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The Acid Test 


What more exacting test can there 


be of dictionary leadership than | 
the judgment of the educational | 


world ? 
Every State that has adopted an 


unabridged dictionary for use i 
its schools has chosen exclusively 


WEBSTER’S NEW 
INTERNATIONAL 





DICTIONARY 


The Colleges voted overwhelm. 
ingly in favor of Webster as the 
standard of pronunciation, in 


answer to questions submitted by | 


the Chicago Woman’s Club. 


Nearly 100% of all schoolbooks are based 

on the New International for pronuncia- 

oe spelling, compounding, and division 
wo 


The New International has been univer- 
sally accepted by those best fitted to judge. 


It is in accord with 
the best modern 


Use it in your own 
schools. 


Write for our 
free booklets 
of interesting 
ways to 

the use of the 
dictionary. 
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A STANDARDIZED TEST OF PROSE APPRECIATION 
FOR SENIOR HIGH SCHOOL PUPILS 


HERBERT A. CARROLL 
University of Minnesota 


Literature appreciation is a very complex ability, based upon 
information, sensitivity to style, sympathetic interpretation of fine 
shades of emotion, intellectual understanding of the deeper meanings 
of a selection, and an interest in plot complications. But greater 
than any one of these—perhaps because it includes all or most of them 
—is the power to differentiate the good from the less good, and the less 
good from the very bad. The high school student who recognizes the 
superiority of a passage from Willa Cather over a passage from one of 
the less literary news-stand magazines possesses discrimination, or the 
ability to appreciate. It was upon this premise that the writer set 
about the task of constructing a test of prose appreciation. 


SELECTION OF TEST MATERIALS 


In choosing selections for inclusion in the test, three factors were 
considered: Subject, length, and source. If several selections are to be 
compared with one another, then they should be on the same topic. 
Seventy-two topics were chosen, and eventually seventy-two sets of 
selections, four in each set, built. 

It was thought desirable, though not absolutely necessary, to have 
the passages short and approximately equal in length. Those used 
were taken from four sources: First choices, from books generally 
considered to be excellent; second choices, from books generally con- 
sidered to be of poor quality; third choices, from the least literary 
magazines; fourth choices, from mutilations. All of the above selec- 
tions were originally chosen with the advice and cooperation of a liter- 
ary critic. A sample set (No. III in the test) follows: 
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AN INTERIOR! 
A 


I went with the little maid into a gorgeously decorated bedroom, all of cream 
color and light blue that blended prettily. The bed was a great, wide affair of 
beautifully carved and ornamented wood, painted creamy white with blue and gold 
trimmings. There was a wonderful bureau and a dressing table to match, and in 
one corner of the room a mirror that went from floor to ceiling. I had to hold my 
breath. 

B 


Lollie had never seen such a pretty room, and it made her gasp to see how pretty 
the furniture was, as well as how pretty the rugs were, and the curtains at the 
windows and the pictures on the wall, but what she really liked best was that 
furniture, for it looked comfortable as well as pretty, and she knew it must have 
cost hundreds and hundreds of dollars. She wished she could live and die in that 
one room, it was so pretty. 

Cc 

An air of Sabbath had descended on the room. The sun shone brightly 
through the window, spreading a golden lustre over the white walls; only along the 
north wall, where the bed stood, a half shadow lingered . . . The table had been 
spread with a white cover; upon it lay the open hymn book, with the page turned 
down. Beside the hymn book stood a bowl of water; beside that lay a piece of 
white cloth . . . Kjersti was tending the stove, piling the wood in diligently .. . 
Sorine sat in the corner, crooning over a tiny bundle; out of the bundle at intervals 
came faint, wheezy chirrups, like the sounds that rise from a nest of young birds. 

D 

Major Prime had the west sitting-room. It was lined with low bookcases, full 
of old, old books. There was a fireplace, a winged chair, a broad couch, a big desk 
of dark seasoned mahogany, and over the mantel a steel engraving of Robert E. 
Lee. The low windows at the back looked out upon the wooded green of the 
ascending hill; at the front was a porch which gave a view of the valley. 


VALIDITY 


The most difficult requirement which the psychologist must satisfy 
in constructing an objective measuring instrument of such an intangi- 
ble variable as appreciation is that his test be valid—that it measure 
what it purports to measure. The Carroll Prose Appreciation Test 
depends for its validity upon two criteria: first, source; second, expert 
opinion. The first of these has already been touched upon. The 
assumption is made that a selection, representative of the art of 
Sigrid Undset, would be superior to a typical selection from a wood- 
pulp magazine, or that a description of spring from Tolstoi could 





1 Key: First, C: Rolvaag, O.: “Giants in the Earth.’”” New York: Harper & 
Bro., 1927, pp. 243-244. Second, D: Bailey, T.: “The Trumpeter Swan.” Phila- 
delphia:; Penn. Pub. Co., p. 26. Third, A: Anonymous, Love-starved. True 
Experiences, Macfadden Pub., Nov., 1930, p. 27. Fourth, B. 
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easily be distinguished from one badly written by deliberate intent. 
To be sure, the passages are short; but a sensitive reader can, even in a 
few lines, discover an incorrectly used word, an inartistic combination 
of phrases, or a faulty selection of detail. If only a single note is 
flatted by a singer, the musician hears it; “errors,’’ if they may be 
called such, in prose are not sosnoticeable, but it is at least possible to 
make gross discriminations. 

The writer intends neither to underestimate nor to overestimate the 
value of source as a criterion of the validity of the test under discussion. 
In the great majority of instances it would probably be, of itself, 
sufficient. Certainly, if the only way that a person can tell the differ- 
ence between the writings of Mr. ‘X” in ‘Hot Dog” and Edith 
Wharton in ‘‘Ethan Frome’ is by having the two carefully labeled 
with the name of the author and a statement of the place where the 
material appears, then that person can hardly be said to possess 
discrimination in the field of literature. It reminds one of the paint- 
ings that have been overlooked because they had not previously 
been identified as the work of one of the great artists. 

However, in a scale of this kind there may be selected occasional 
passages of very nearly equal value, since at times an inferior author 
succeeds in writing a few paragraphs that are excellent, while a writer 
of great merit drops to mediocrity. Largely because of the existence of 
such a situation, the criterion of source has been supplemented by the 
criterion of expert opinion in determining the order of merit of the 
passages in the several sets. Only those selections on which there has 
been, among the sixty-five judges, a consensus of opinion in absolute 
agreement with the source are included in the test. (See Table I.) 

It will be seen from Table I that the writer made use of three groups 
of judges; critics, members of University English staffs, and high school 
teachers of English. Listed as critics are book reviewers, authors of 
fiction, and librarians. Numbered in this group are men and women 
internationally known in the world of literature. 

Unfortunately, it was impossible to have the same judges for all 
twelve sets, due principally to the fact that the test was revised 
several times. A total of twenty-five rendered judgments on all of the 
selections appearing in the final test; one group of sixty-five judged 
sets I, III, VI, VIII, IX, and X; another group of sixty-five—with the 
above-mentioned overlapping—judged sets IV, V, VII, XI, and XII; 
while a third, made up of members of these two groups, judged set II. 

In order to see if the critics, university instructors, and high school 
teachers would agree, as separate groups, with the criterion of source 
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and with one another, their judgments were segregated and tabulated. 
The results appear in Table I. Even with so great a reduction in 
numbers, with the attendant increase in possibilities for error and 
disagreement, there is unanimity among the three groups and source on 
forty-one of the forty-eight selections. Of the remaining seven, there 
are four tie votes and only three disagreements. This appears to the 
writer to be a rather striking correlation. 

It will be seen that it is the consensus of the experts’ judgments that 
is being accepted. In some instances the discrepancy between an 
evaluation by a single expert and the criterion of source is as great as an 
interchange of a first and third choice, or even of a first and fourth. 
This may be the result of an actual difference of opinion, or, what is 


more likely, of a hasty rating or even of careless errors in recording 


decisions. 

The validity of the test, then, rests upon the criteria of source and 
expert opinion. Ifa high school student agrees, in his judgments, with 
the source and with the consensus of the rating given by the experts, he 
is said to be correct—to possess the capacity to appreciate prose 
literature. If he does not agree, he is said to be wrong. 


RELIABILITY 


Table II explains itself. It is interesting to note that the two 
reliability coefficients are the same, 7.e., .71. This is much higher, 


TaBLE IJ.—RELIABILITY COEFFICIENTS FOR SENIOR HiagH ScHoo.t Popits 








Re-test Split-halves 
Mititidenondudédesatdenekasdveneseon 467 300 
Pk devccrnntbstanus cseenenwe .71 + .016 .71 + .019 











insofar as the writer has been able to discover, than that of any other 
test of literature appreciation. It may be compared, for example, 
with the .44 published for the Abbot-Trabue Poetry Test. 


ORDER OF DIFFICULTY 


The twelve sets of selections which make up the total test have been 
arranged in order of difficulty. Percentages appear in Table III. It 
will be readily seen that the intervals of difficulty between the several 
sets are by no means equal. To have made them so would have 
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required more time and labor than would seem justified by the possible 
slight increase in reliability and smoothness of the distribution curve. 


TasLE III.—Orper or DirricuLty oF THE TWELVE Sets or SELECTIONS AS 
DETERMINED BY PERCENTAGE OF CorRRECT Scores MADE BY THREE 
HunprepD_ Hie Scuoo. Poprits 


ee. 


I II III IV V VI 





VII | VIII 








IX | X | XI | XII 






































58.9 | 58.1 | 57.3 | 53.6 | 53.3 | 52.6 47.1 43.7) 42.4/| 41.0| 37.8) 35.8 








ADMINISTRATION AND SCORING 


Detailed directions for the administration and scoring of the test 
appear in the Manual. Only two points need be considered here. 

First, there is no time limit, as it was felt that it would be unwise to 
try to put a stop-watch on appreciation. Nearly all high school 
students will finish within the usual forty minute period. When the 
test was standardized only one hundred eighty-seven pupils in over 
three thousand handed in incomplete record sheets, the time allowed 
being a single period of forty minutes. In order to learn whether or not 
these one hundred eighty-seven cases would have seriously affected the 
final distribution of scores, they were matched on the basis of Grade 
and School with one hundred eighty-seven children, chosen at random, 
who had completed the test. The papers were then scored on the basis 
of the first half only, and the means compared. (See Table IV.) 


TaBLE IV.—A CoMPARISON OF THE DISTRIBUTION OF SCORES ON THE FIRST 
Haur oF COMPLETE AND INCOMPLETE RECORD SHEETS 

















Completed Incompleted 
records records 
aa ahah aiid. oie ethan i Suk S eke Mee ALS | 187 187 
|! ee! en ae! lr 18.12 + .312 
irik Bel nd 00s 04 ccs a beedeeneedrevassess 6.22 6.32 
| 





Moom. — Mincom. = 19.08 — 18.12 = .96 + .438 





Since the difference between the means divided by its probable error 
is only a little over two, it was concluded that these incomplete records 
could be thrown out without the final distribution being appreciably 
affected. 
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The second point concerns scoring. Several methods were experi- 
mented with, the one giving the highest reliability being that of 
weighting the first and fourth choices. Viewed subjectively, as well, 
this seemed to be the most satisfactory. The method used, then, 
places a premium upon the ability to recognize the very best and the 
very worst; or, to put the latter part of the above statement negatively, 
penalizes an individual for inability to recognize the almost complete 
lack of merit in the mutilations. Numerically, two points each are 
given for the correct ranking of selections I and IV, and one point each 
for the correct ranking of selections II and III. 


DISTRIBUTION OF SCORES 


In its preliminary and final forms the Carroll Prose Appreciation 
Test has been given to approximately nine thousand people of varying 
ages and educational levels. It is a product, then, of a series of trials 
on actual school populations. In all, seventy-two sets were used at 
one time or another; of these, twelve remain. This final form has been 
standardized for Grades X, XI, and XII. The subjects used were 
children in the four following high schools: Mechanic Arts, Central, and 
Humboldt, all of St. Paul, Minnesota, and the University High School 


TABLE V.—DISTRIBUTION OF ScorES MaAprE By SENIOR HiGH ScuHoou PvopIis 








Scores Grade X Grade XI Grade XII Total 
60-63 0 1 2 3 
56-59 5 6 17 28 
52-55 10 30 44 84 
48-51 33 54 90 177 
44-47 66 104 118 288 
40-43 133 134 150 417 
36-39 153 177 171 501 
32-37 143 —C 148 153 444 
28-31 139 126 108 373 
24-27 117 97 76 290 
20-23 105 63 37 205 
16-19 59 | 40 23 122 
12-15 27 14 7 48 

8-11 8 5 3 16 

4-7 1 1 1 3 
0-3 1 0 0 1 

N 1000 1000 1000 3000 
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at the University of Minnesota. The distribution of the scores made 
by three thousand of these pupils appears in Table V. 
In Table VI appear data relative to the central tendency and 


variability of the scores made by several groups. 


TaBLeE VI.—Measvres INDICATING THE DISTRIBUTION OF THE ScorES MADE BY 
THE SEVERAL Groups THAT TooK THE FINAL FORM OF THE CARROLL 
Prose APPRECIATION TEST 











High school pupils 
College 
Measures Grade | Grade | Grade Total students Judges 
x XI XII 7 
ai teen a hie ae ie eas 1000 1000 1000 3000 225 25 
H.P.8 jesse 72 72 72 72 72 72 
ee 6-60 5-63 1-63 7-65 34-64 
M (with PE).......... 32.83 35.56 38.00 35.45 43.32 53.00 
+.204 | +.202 | +.201 | +.120| +.425| +.707 
ib dita aha A ain te he aL ea 9.56 9.48 9.41 9.72 9.45 5.24 
M. (with PPIE).......2- 33.22 36.18 38.32 35.98 43.55 54.33 
+.255 | +.252 | +.251 | +.150 | +.531] +.884 
Deine dé ee eames 39.90 42.40 44.58 42.38 50.11 57.25 
re re eee Poe eee 25.41 28.94 31.79 28.69 37.34 49.25 
sitadscedk oheceawans 7.24 6.73 6.40 6.84 6.38 4.00 























Besides presenting facts concerning the distribution of the scores, 
this table gives additional proof of the validity of the test. It will be 
noted that there is a steady increase in central tendency from Grade X, 
through college, to the judges. - Table VII shows clearly that the 
existing difference between the means of any two groups is a highly 
reliable one. The consistency of the differences with respect to high 
school students is shown in Figure 1. 


TaBLE VII.—DIFFERENCES, WITH THEIR PROBABLE ERRORS, BETWEEN CERTAIN 
Pairs oF MEANS 


M(sudges) — Mcot. students) = 53.00 — 43.32 = 9.68 + .825 
M co. Be Maa. 8. Stud.) = 43. 32 — 35. 45 = 7. 87 + .442 
Mor. xu) — Moar. x1) = 38.00 — 35.56 = 2.44 + .285 
Maar. x) — Mar. x) = 35.56 — 32.83 = 2.73 + .287 


SUMMARY 


The Carroll Prose Appreciation Test for senior high school students 
is based upon the assumption that the ability to appreciate literature 
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can be measured by revealing the degree to which an individual 
discriminates among passages of varying worth. Its validity has been 
determined by two criteria: source and a consensus of the judgments of 





~.—. Grane X 
100 


90 














0 10 20. 30 §o 50 63 
SCORE 


Fic. 1.—Percentile Distribution of Scores Made on the Carroll Prose Appreciation Test 
by 1000 Children from Each of the Senior High School Grades. 
sixty-five experts, the correlation between the two being perfect. Its 
reliability, found for four hundred sixty-seven senior high school 
students, is .71 + .016. The amount of time needed for taking the 
test is approximately thirty-five minutes. Tentative norms have 
been established on three thousand high school pupils. The publisher 

is: Educational Test Bureau, Minneapolis, Minnesota. 
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THE RELIABILITY OF A SINGLE TEST ITEM 


KARL J. HOLZINGER 


The University of Chicago 


INTRODUCTION 


In this paper we shall introduce formulae giving the standard error 
of response or measurement of a single test item. The response error of 
series of items is already known. The problem, then, is to find this 
error for a single item and show the relation to the standard error of a 
series of such items. 

It is hoped that these new formulae will be of some value in building 
up tests of items of known reliability and in predicting the final reliabil- 
ity of a number of such items when combined. Conversely it is 
believed the formulae may be useful in appraising the reliability of 
tests of different lengths by reducing the measure of reliability to the 
average item basis. 


THE ERROR OF RESPONSE OR MEASUREMENT OF A LENGTHENED TEST 


Before introducing the new formulae we shall examine the effect of 
test lengthening upon the usual formula 


Oe = oV1 —ru (1) 


This formulae may be derived in several ways, one of which is as 
follows: 
Let z; and z; = scores on two forms of a test, expressed as devia- 
tions from respective means, 
ri: = reliability coefficient of these forms 
o, and o; = standard deviations of such scores 
z, = ‘‘true” score on either form (average of large 
number of testings) 
é, and e; = uncorrelated errors defined by the expressions 


Zi = Te + 41 
Zi = Xp + 1 


Subtracting, member by member, gives 
Ai— 71 KH Cl — C1 


Squaring, summing, and dividing by N (number of cases) gives 


o3? — 2ri0101 + 1? = o*,, + o*., 
411 
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Ifo; = or, ando,, = o-, we then have 


ge? = 01°(1 — rit) (2) 


a, is thus the standard error of response of (x; — z,,). 

We shall inquire next into the effect of test lengthening upon 
formula (2). The reliability of a lengthened test is given by the well- 
known Spearman-Brown formula 


Nr it 


me Tt — Bre (3) 





The standard deviation of a lengthened test is expressed by the easily 
derived formula 





on, =oi nt n(n — l)ru (4) 
Writing the standard error of response of the lengthened test as 
noe = nV 1 — Tan 


we have, upon substituting formulae (3) and (4), 








nTe 
Vn 


In the derivation of the last three formulae it has been assumed that 
the intercorrelations of the component tests are all equal, whether on a 
single form or between any two components on similar forms of the 
tests. Thus if the components on each form be designated as 'z,, 
271, 941, ... "1 and ‘a1, 721, *x1, .. . "x1 respectively, all of the 
n? intercorrelations, 2‘x;‘z:/No.01 (¢ = 1 to n) are assumed equal. 
The standard deviations of ‘x; and ‘xz; are also assumed equal. 

If two forms of a test have been prepared so that component 'z; is 
very similar to 171, component 2z, is very similar to *z; etc., then the 
correlations between ‘z, and 'z; or between 7x; and 2z; will usually be 
higher than the cross values between 'z; and 2x; or between 2z, and 
1y;, etc. We shall therefore distinguish two types of such coefficients, 
viz., n parallel values 711 between ‘x; and ‘x: or x; and 2z1, etc., and 
n(n — 1) cross values designated as .ri1 between 'z, and *z; or between 
27, and \z, ete. With this distinction formula (3) would be written 


(3') 





= oiV1 — Tn * Oe (5) 


_ Mtn — 2(n — 1) 





_— n+n(n — 1)ru 
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Formula (4) would be expressed as 





On = Oi1Vn+ n(n — 1)-ru (4’) 
and formula (5) would become 
7 =o1V1 — orn (5) 


Formula (5) is important because it shows that the response error 
is a function of the length of a test. We cannot, therefore, make 
adequate comparison of the response error in tests until their varying 
length has been taken into account. Thus if a test of 25 items has a 
response error of ¢,, this error will be increased to 2c, upon increasing 
the test to one hundred items, etc. 

Formula (5’) shows that in obtaining the response error of a 
lengthened test we need consider only the direct component reliability 
coefficients ,r:, of which there areninnumber. This greatly simplifies 
the problem because all of the n(n — 1) cross coefficients may be 
ignored in obtaining the value ,jo.. 

If the reliability of a lengthened test is required formula (3’) will 
give much more accurate results than formula (3) if the parallel and 
direct coefficients differ greatly in value (as they often do). The com- 
ponent standard deviations should also be taken into account, but the 
complexity of the resulting formulae makes this unpractical. We shall 
therefore assume these all equal with a certain undetermined error 
resulting. 


THE ERROR OF RESPONSE OF A SINGLE ITEM 


In order to obtain an expression for o, which will apply to a pair of 


equivalent items we shall first express formula (1) in a different form: 
Let 
d= 2 — 7; 
and 
D = X, — X, for raw scores with means M, and M,. 


From these last expressions we find that 


oc. = 2017(1 7 T11) = 20,” (6) 
and 


2 
oat = = — (My— Mi) (7) 
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Therefore 


2 
ot = 3] 2D" — (an, - My] 8) 

Formula (8) will next be applied to the case of a pair of single items, 
one for each form of a test, and each scored either 0 orl. If N pupils 
try both of these items, the scores on the pair of items are recorded as 
(0, 1), (1, 1) (0, 0) or (1, 0). The number of pupils getting these four 
possible pairs of scores will be denoted by a, b, c, and d respectively as 
illustrated by the following scheme: 





Score on | a| bd 


Form X, | c|d 
Score on Form X,; 











Now it is evident that D = X, — X, can take only the four values 
(0, —1), (1, —1), (0, —0) and (1, —0) so that 2D? =a+d. Itis 
d—a 


N 


»_Watd_ a — d\’ 
E-G] 


as the desired standard error for a pair of items. Ifa = d this formula 
reduces to 





also apparent that M, — M, for the above table = 
We thus find 





10" os y (10) 
If (a — d)?/N is negligible but a — d ~ 0 we may write 
Ifa+d 
10 e" -_ | N (11) 


The reliability coefficient for the pair of items may also be found 
from the point correlation coefficient 


be — ad 




















71 = (12) 
V(a + b)(e + a)(a +c) + d) 
and the standard deviations are given by 
_ V(a+c)(b +d) 
10, = /N (13) 
7 =< Mat bye + 4) 4 
19; UN (14) 
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These two standard deviations have been assumed equal in formula (9), 
so that b = c. The correlation (12) may then be written 








b? — ad 
= GEO +E) we 
If a = d this formula is further reduced to 
Wisi 
mn =F as (16) 


As a check on the appropriateness of the point correlation for these 
purposes we may solve the expression ¢,? = o,? (1—r,;) or 


1 #? - (tr) ]- (a + b)(c + d) 





a1 N N N? (1 — riz) for ri; 
The solution gives formula (15). 


THE RELATION OF THE ERROR OF A PAIR OF ITEMS TO THAT OF A 
LONG TEST 


Returning to the formula 


n0e 


/n- V1 — ou (5’) 


we may now regard n as the number of items in the test and write 


Fenda’ -(Ce)] <I] an 


This formula may be used to calculate the error of estimate of a 
long test composed of items the error of which has been found by for- 
mula (9). If the errors of the individual items are all equal the for- 
mula is exact; if the errors are not equal their average may be taken as a 
good approximation in obtaining a value for ,c, as will be shown below. 

As a theoretical illustration assume one hundred test item pairs 
each of response error 0.2 as determined by formula (9). The standard 
error of response of a test of one hundred such items (two forms one 
hundred items each) would be predicted by formula (17) to be two. 

Conversely, if a test has a response error of two and is composed of 
one hundred items, formula (5’) shows that the standard error of each 
item pair is 0.2 (assuming these latter errors equal). It will be shown 
below that formula (5’) gives a good measure of the standard error of 
the average test item pair when these errors vary considerably. 
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Two suggested uses of the above formulae are: 

1. Given the standard error of response of long tests composed of 
items scorable 0 or 1 formula (5’) gives a good measure of the average 
response error of test item pairs. This makes possible the direct 
comparison of the response errors in tests on the item basis. Thus if 
one test furnished a value from formula (5’) of 0.2, while another test a 
value of 0.4, we can say the former is composed of better items in the 
sense that their average response error is less. 

2. In building a test, two forms may be prepared and given to a 
group of pupils. The values of ., may then be calculated from for- 
mula (9) or (11) and a group of pair items selected for the long test on 
the basis of small average response error. From these chosen items the 
standard error of the proposed long test may be calculated by formula 
(5’) or (17). Permissible variations in ,c,? are discussed below. 


THE SAMPLING ERROR OF 0,2 


Formula (11) indicates that io, is approximately a proportion, so 
that its standard error of sampling is given by the formula 





10-7(1 kn 10") 


N 





SD (sampling of 10,7) = J (18) 
It is now suggested that in building a test as described in the pre- 
ceding section, variations of ,o.? about the average o,? be used within a 
range of +3SD (sampling of the average i0,”). Experimental verifica- 
tion of this proposal is given in the next section. 
We shall take as an approximation to the standard error of the 
average value of ,o.” the quantity 





o.2(1 — o,2) 


SD (sampling error of ¢,2) = m Nn (19) 





where o,2 denotes the average value. 


EXPERIMENTAL VERIFICATION 


A number of the writer’s students are trying out test material with 
the purpose of verifying the above formulae. One of these experiments 
is briefly stated below: 

Two forms of Beauchamp’s Science Tests of fifty items each were 
given to one hundred five pupils. All items were scored 0 or 1 and 
fifty values 144[(a + d)/N] calculated. The average value of 0.’ 
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from formula (11) was 0.15 +0.005. The reliability of the fifty items 
was 0.837 with o, = 7.84 which gives a value of 0.200 when substituted 
in the formula o,2(1 — ran)/n. The difference between 0.20 and 
0.15 + 0.005 indicates that the discrepancy exceeds sampling varia- 
tion and that formula (17) may not be applied to such data. 

The discrepancy is probably due to the wide variation in values of 
4[(a+d)/N]. To test this hypothesis we next selected twenty-two 
items with reliability exceeding 0.35 by formula (12) and not differing 
from one another in sampling error by more than twice the value from 
formula (18). For these twenty-two pairs of items the average value 
of ,.? from formula (11) was 0.13 + 0.01. The reliability of the 
twenty-two items was 0.813 with o, = 4.01 which gives a value of 0.14 
when substituted in the formula o,?(1 — ran)/n. The difference 
0.14 — 0.13 + 0.01 here indicates that formula (17) applies very well 
to the data. 

The suggestions from this experiment are that the assumption 
underlying formula (17) as applied to average values of 14[(a + d)/N] 
are met if variations do not exceed two standard deviations by formula 
(18), and if items of highest reliability are selected. 

The above twenty-two pairs of items were also used to illustrate the 
use of formula (3) and (3’). The average reliability of these twenty- 
two items was 0.412. If this value be substituted in formula (3) for 
n = 22 we find r,, = 0.939 + 0.019 (Shen’s formula). This predic- 
tion exceeds the observed reliability coefficient 0.813 by an amount 
beyond the bounds of sampling error. The formula does not apply 
because the cross correlations were assumed equal to the parallel values. 
When the former are calculated and used in formula (3’) we find 
Tan = 0.847 + 0.022 which does not differ significantly from the 
observed value 0.813. Clearly formula (3’) should be used to estimate 
the reliability of a lengthened test where parallel and cross coefficients 
differ. In this case the average of the parallel coefficients ,r:; was 
0.412, while the average of a sampling of the n(n — 1) cross coefficients 
‘yy was only 0.158. 

The length and detail of this paper make it desirable to defer further 
illustrations until later. It is hoped that those interested in test con- 
struction will also check these formulae with experimental data. The 
writer will also appreciate criticism because he is well aware of several 
doubtful points in the application of the formulae. 
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A METHOD FOR DETERMINING THE ABSOLUTE 
AFFECTIVE VALUE OF A SERIES OF STIMULUS 
SITUATIONS 


PAUL HORST 
Personnel Research Division, United States Civil Service Commission 


I. INTRODUCTION 


The object of this investigation was to develop a technique whereby 
a series of stimulus situations could be scaled with reference to their 
affective value. By ‘‘affective value’? we mean simply the degree 
to which the stimulus situation evokes feelings of pleasantness or 
unpleasantness. The method assumes that there is a continuous 
gradation from those experiences which are extremely unpleasant to 
those which are extremely pleasant. 

For example, let us imagine a scale as indicated in Fig. 1: 


| 
O 





| | 
U P 


Fig. 1. 
We assume that any stimulus situation may be allocated on this scale 
according to the degree of affect which it normally evokes in an individ- 
ual. In the following analysis we attempt to measure the distance 
which any given situation has when allocated with reference to its 
affective value, from the point of zero affect. This distance is to 
be measured in terms of units to be subsequently defined. 

The statistical technique involved is a modification and extension 
of a method developed by Thurstone.! The technique makes possible 
the determination of scale separations, on a psychological continuum 
of items in a series of dissimilar stimulus situations. So far as the 
method is concerned, these situations may be anything, providing they 
can be compared with reference to the degree of affect which they 
evoke. | 

Evidently any given stimulus situation will not evoke the same 
degree of affect in all individuals, nor will it evoke the same degree 
of affect in a given individual from one time to another. Without 
committing ourselves as to the manner in which an individual evaluates 
the affective value of a stimulus situation or distinguishes between the 


relative affective values of two or more situations, we may assume that 
418 
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some one affective value is associated more frequently with a given 
situation than is any other value. 

For convenience we employ the terminology of Thurstone. He 
defines the ‘‘discriminal process” as that process by which the organ- 
ism indentifies, distinguishes, discriminates, or reacts to stimuli. The 
fluctuation in the discriminal process for a uniform repeated stimulus 
is designated the “discriminal dispersion.”” The discriminal process 
which is associated more frequently with a given stimulus situation 
than with any other situation is termed the ‘“‘modal discriminal 
process’”’ for that stimulus situation. Thurstone shows that the funda- 
mental psychophysical equation for determining the scale separation 
between two stimulus situations is 


S; _ S; = bi 07? + o;” ™ 27 530 0; (1) 





where 

S; is the modal scale value on the psychological continuum for the 
stimulus situation 7. 

S; is the modal scale value on the psychological continuum for the 
stimulus situation 7. 

b;_; is the sigma value as found in a table of the normal probability 
integral for the experimentally observed proportion of judg- 
ments ‘‘z less than 7.” 

o; is the standard deviation of the discriminal dispersion of situation 

7 on the psychological continuum. 
g; is the standard deviation of the discriminal dispersion of situation 
j on the psychological continuum. 

r;; is the correlation between the discriminal deviations for situa- 
tions 7 and j. 

The assumptions upon which the validity of the above equation 
rests are given as follows: 

(1) Every stimulus in the stimulus-series is associated with a 
modal discriminal process with which the organism identifies the 
stimulus for a prescribed attribute. 

(2) The modal discriminal process for any given stimulus retains 
at least some of its identity even when the stimulus is combined with 
other stimuli into a single perceptual judgment. 

(3) The modal processes may be arranged in a linear psychological 
continuum in the same serial or rank order as the corresponding 
stimulus series. 

(4) In addition to arranging the discriminal processes in rank 
or serial order, linear separations between them are assigned on the 
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assumption that the discriminal dispersion for any stimulus is normal 
on the psychological continuum. This assumption is subject to 
experimental verification. 

The meaning of equation (1) may be indicated diagrammatically 
by Fig. 2: 










f 
BS) 
| 








ocean essen em 





i — 
U S: S; 0 "Pp 
Fig. 2. 





The base line of this figure is the affective scale shown in Fig. 1. 
We have given two situations 7 and j7. For each situation we have 
a number of different discriminal processes identifying the situation 
with respect to its affective value. These processes may differ because 
they come from different individuals, or because they came at different 
times from the same individual, or both conditions may contribute to 
the variation. In any event, we obtain a distribution of discriminal 
processes on the affective scale or continuum for both situation 7 and 
j. According to assumption (4) the discriminal dispersions of the 
two situations respectively are normal. The mean or modal scale 
value S; is by definition the affective value of situation 7. Similarly 
S; is the scale value of situation 7. The standard deviations of the 
discriminal dispersions of the two situations are o; and a; respectively. 

From equation (1) it is clear that the value b;_; is a function both 
of the distance between S; and S; and the dispersions of the two 
distributions. | 

If, in addition to the four assumptions previously made we assume 
that: | 
(5) The correlation between the discriminal deviations for any 
two items in the series is zero. 

(6) All of the discriminal dispersions in the series are equal. Then 
equation (1) simplifies to 


S; —_ S; = b;_ 0/2 (2) 


Thus far no unit has been indicated for measuring the distances on 
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the affective scale. If we adopt oc, or the standas‘l deviation of the 
discriminal dispersion, as the unit of measurement, then equation (2) 


becomes 
S;— 8; = b V2 (3) 


It should be noted, however, that equation (3) zives only the dis- 
tance between S; and S;. It does not give the dittance of either S; 
or S; from the origin of the scale, that is, from the pilint of zero affect. 


II. THEORETICAL 


1. The Difference Equation—We shall now derive equation (3) 
analytically. 

In Fig. 2, S; has been defined as the modal Aas value on the 
psychological continuum for situation?. Since the ‘liscriminal disper- 
sion is, by assumption, normal on the continuum, the equation to the 
distribution curve is | 

_ — 5 


f(a) = Fee : (4) 





where 

xz is the affective value corresponding to a yiven discriminal 
process measured from the point of zero affect in units of the dis- 
criminal dispersion. 

S; is the modal scale value measured from the poir', t of zero affect in 
units of the standard deviation of the discrimina’ dispersion. 

Analytically, S; is that value which in the equation 
z— Si 


f(x)dzx . (5) 


makes P = 0.5. 

Since equation (5) is merely the expression for the proportion of 
the area of the curve below the ordinate erected at z, it is obvious that 
P = 0.5 when x = S,. 

Suppose we represent the distribution of discriminal processes 
associated with situation 7 by Fig. 3. Then the probability that a 
discriminal process corresponding to a value below x will be associated 
with situation 7 is given by the proportion of the ares below the ordi- 
nate at z. If this proportion is given we can substitute it in equation 
(5) and find the corresponding value of (« — S;) from a normal 
probability table. Designating this value by (d) we have 


z—d=S8; (6) 
where both z and dare given. Hence S; can be determined. 
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Theoretically, then, we might determine the affective value of 
stimulus situation 7 by selecting any arbitrary value of z and obtaining 
from a large group of individuals the proportion of the group which 
judges the affective value of situation 7 to be less than z. Substituting 
this proportion in equation (5), (d) in (6) could be determined and S; 
solved for. 

Obviously, however, the procedure is not feasible since it would be 
impossible for individuals to agree on the meaning, in terms of affective 
value, of any given numerical value. Therefore, instead of dealing 
with distributions of absolute processes, we deal with distributions of 











k—d 
x S; 
Fig. 3. 


differential processes. By an “absolute process’? we mean a dis- 
criminal process which identifies a single stimulus, whereas, the term 
“‘ differential’ is used to designate a discriminal process which differen- 
tiates between two stimuli. 

We let x; and z; represent the corresponding affective values of 
the absolute discriminal processes by which a given individual identi- 
fies situations 7 and j, and x;_; the affective value corresponding to the 
differential process by which he differentiates them. 

If the distributions of x; and xz; are normal, then the distribution of 
z;_; is also normal. Hence, instead of representing two separate 
distributions of absolute processes on the psychological continuum, 
as in Fig. 2, we postulate a single distribution of differential discriminal 
processes. This is represented diagrammatically in Fig. 4. 

Now S; — S; is defined as that value for which the probability is 
0.5 that a differential discriminal process associated with situations 7 
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and j will lie below S; — S;. Similarly, the probability in also 0.5 that 
the process will lie above S; — S;. Hence S; — S; is that value which 
makes P = 0.5 in the equation 


zi_j— (Si— Sj) 
Pej = | f(tidz; (7) 
In general, if we have given the probability that a differential 
process associated with situations 7 and j will lie below any value 
z;-;, we can find the distance between the value z;_; and S; — Sj. 
The probability that a differential process corresponding to a value 






K-55 











ij O S75 
Fic. 4. 


below x;_; will be associated simultaneously with situations 7 and 7 is 
given by the proportion of the area of the curve below the ordinate 
at a;; This area is represented by the cross-hatched portion of 
Fig. 4. If this proportion is given we can substitute it in equation (7) 
and find the corresponding value of z;; — (S; — S;) from a normal 
probability table. Designating this value by b;_;, we get 


ti; — (Si — S;)) = b-;V2 (8) 


where the value +/2 is the standard deviation of the distribution of 
differential processes in units of the discriminal dispersion of a dis- 
tribution of absolute processes. 

If we obtain from a large group of individuals the proportion of 
the group who judge the affective value of situation z to be less than 
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situation j this means that they judge S; — S; to be less than zero. 
In this case z;.; = 0. Substituting this value of z;_; in equation (8) 
and remembering that b;_; is negative when S; < S; and positive 
when S; > S; we get 


S; — 8; = bi 2 (9) 
which is the same as equation (3). The value };_; is represented in 
Fig. 4 by the distance between 0 and the mean S; — Sj. 
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2. The Sum Equation.—It is evident from the preceding section 
that in order to find the scale separation between two stimulus situa- 
tions these situations must be close enough together on the psychologi- 
cal continuum so that one will not be judged greater than the other 
by all members of the group. The possibility of determining scale 
separations depends on the fact that there is disagreement as to which 
of two stimulus situations has the greater value. This principle 


A\/P 7 


U S; 0 x P 
Fic. 6. 

















has been represented by the overlapping distributions of discriminal 
processes in Fig. 2. 

But suppose we have a situation as indicated in Fig. 5. 

Here again we have the same affective scale as in Figs. 1 and 2. 
The meaning of the distributions is the same as in Fig. 2. But here) 
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is a very unpleasant situation and 7 is a very pleasant situation. It 
is highly improbable that anyone would regard j as a more pleasant 
situation than 7. Since there is not this disagreement, the method of 
the preceding section cannot be employed to find the distance between 
S;and S;. If, however, we had a number of situations which could be 
represented by a series of overlapping distributions along the entire 
range between S; and Sj, as indicated by Fig. 6, 

then the method of the preceding section would be available. We 
could determine the scale distance between the means of each adjacent 
pair of distributions and by adding all of these obtain the distance 
between S; and S;. 

As a matter of fact, this is a part of the procedure actually em- 
ployed. But, as will be shown later, this procedure if employed 
alone enables us to obtain only scale separations, whereas we are 
primarily interested in finding the distance of each scale value from 
the origin. For this reason, we develop a method for determining 
the sum of the scale values of two situations. 

First we define a “‘summative process” as that process by which 
the organism identifies the summative value of two stimulus situations. 
We let z;,; represent the affective value corresponding to the summa- 
tive process by which the individual integrates situations 7 and j. 

Retaining the assumptions of the previous section, we can write 
an equation analogous to (7), namely, 


zi+j— (Si+8)) 
Pini = | S(eiss)dri4; (10) 


where S; + S; is the algebraic sum of the values S; and S;, and is 
defined as that value which makes P = 0.5. 

To determine this value experimentally, the same logic is employed 
as in the previous section. We obtain the proportion of individuals 
who judge the summative value of situations 7 and j to be less than 
zero. In equation (10) the value of z,,; in the upper limit of the 
integral is therefore zero. Since P is given, the upper limit —(S; + S;) 
can be obtained from anormal probability table. Indicating this value 
by bi; and multiplying by +/2, the sigma of the distribution, we get 


Si + 8; = bur/2 (11) 


remembering that b;,; is negative when P < 0.5 and positive when 
P> 0.5. 
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III. TECHNIQUE 


1. Calculation of Scale Separations by the Average Difference Method. 
The method employed by Thurstone! in solving for the scale separa- 
tions of stimulus values is as follows: 

Having given the observation equations of the type 


S; — S; = bV/2 


the constant values b;; are arranged in a skew symmetrical table 
thus: 


1 2 3 4 n 
1 0 —bi_» —b;_3 —by4 ees —bi_., 
2 by 0 —_ bo_s — bo_4 coe ™ bo_n 
a, ‘Mes ie... See (12) 
n = bo_» -_ ba_n eee 0 


where 
b;_; is the sigma value of the proportion of individuals who judge 
S; — S; to be less than 0, or, what amounts to the same thing, 
who judge S; to be less than Sj. 

It will be noted that the entry in the z’th row of the j7’th column is 
always the same as the entry in the j’th row of the z’th column with 
sign changed. That is —b;; = +b;;. Only the values below the 
diagonal are given in the observation equations. Those above are 
copied from those below. 

A table of differences is derived from the first table by subtracting 
column 2 from 1, 3 from 2, 4 from 3, etc. Each remainder in a given 
difference column is of the type bix—», — bix so that summing for any 
given column and multiplying by +/2/n we get 


Sx-1 — Sx = vay (—b-cx-1y + bi_x) (13) 


giving the scale separation between S;,_; and S,. 
The same general procedure may be followed when the equations 
are of the type 


\S;+ S; = bi; 2 


In this case, however, the table corresponding to (12) is not skew 
symmetrical, but perfectly symmetrical. That is, 
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22> =. n 
1 Dise Diss Disa fm Ditn 
2 Bite — bois bors aie doin 
3 bi+s bors — bsi4 aia Dain (14) 


n Disn Doin bain Dain a 


This is due to the fact that we are considering sums of scale values 
rather than differences. Furthermore, instead of having a diagonal 
of zeros the diagonal elements are indeterminate, and hence left 
blank. 

In forming the difference table of (14) each subsequent column is 
subtracted from the preceding one, so that each remainder in a given 
column is of the type bistc-» — bi4x. Summing each difference 


ae 
column and multiplying by 7 2 Bives, 


2 
Sx-1 = Sx — a ac bi+x) (15) 


The set of scale separations obtained from (15) may be averaged 
with those obtained from (13). 

2. The Least Square Method.—But while either (13) or (15), or a 
combination of the two, give scale separations, they give no clue as 
to the origin of the scale. Since our fundamental problem is the 
determination of this absolute origin, we must develop a method for 
combining the sets of equations 


S;- S;= bn 2 (9) 
S: + 8; = biur2 (11) 


so that the S’s can be determined in terms of their distance from this 
absolute origin. ‘To determine the scale values in terms of an absolute 
origin a “least squares”’ procedure is employed. The normal equa- 
tions are derived from the observation equations in the conventional 
manner, but due to the form of the observation equations the normal 
equations are much easier to derive than with the ordinary (n)-variable 
problem. The derivation is as follows: 


>(S; — S; — b-V/ 2)? = V = aminimum (16) 
(8; + 8; — bu /2)? = U = a minimum (17) 
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Considering first equation (16), we have 


aV aV aV 

a ~ Sm *- ag 
The general form for any equation (K) is, 
aV 
95 7 51 — Sa - + + Sen —-1)-- > — 8. — 


VS bu - ¥ bx) =0 (18) 


K+1 


There will be (n) of these equations. First we determine the nature 
of the matrix of the coefficients (18), leaving the discussion of the 
formation of the constant terms for a latter paragraph. An examina- 
tion of (18) shows that the matrix of the coefficients of the parameters 
or scale values takes the form 


Ai; = 
n—1 —1 —l--++ -1 
—-1 n-1 —-1l--+ -1 
—1 —-l n-1:--- —-1 (19) 
—l1 -1 -1-++n-1 








But since the number of —1’s in every row of (19) is (n — 1), the 
sum of the elements of every row is zero. According to the theorem 
in the elementary theory of equations that if the sum of the elements 
of every row is zero, the determinant of the matrix vanishes, A;_; is 
zero and there is no unique solution for the constants in equation (18). 
This necessitates a combination with the normal equations derived 
from (17). The method of combining the two sets of equations will be 
discussed later. 

Recurring now to the formation of the constant term in (18), 
it will be noted that the general expression for the K’th constant term 


is 
n K- 
Kr=V 2 Dou _ S b«:) (20) 
1 


K+1 


But we see that (20) is precisely the result obtained if we sum the 
K’th column in table (12) and multiply by +/2, where the summations 
take into account that all the b’s above the diagonal have a negative 
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sign. This procedure enables us to get the respective constant terms 
directly from the table of b’s. 


As indicated, however, equations (18) do not give a solution of the 
parameters, due to the fact that the determinant of the matrix A;_; 
in (19) vanishes. Now let us consider the normal equations to be 
derived from (17). 

Taking partial derivatives with respect to each of the S’s and 
equating to zero gives (n) equations. These are of the form 


wU _g4s, ove + Sx(n — 1) aidaiine +S, —~V/2(Zbe:) = 0 (21) 


OSx 
As in (18) the constant terms of these equations are formed by sum- 
ming the columns in the table of },,,;’s and multiplying by +/2. 

We see that while the constant terms for (18) are given by 


Ki;=V 2 > bx; — Lx) (20) 


K+1 


the constant terms for (21) are given simply by 
Kix; = V2(2bxi) (22) 


The reason, of course, that we have a difference of two summations 
in (20) and not in (22) is due to the fact that while in table (12) all 
the b’s above the diagonal are given the negative sign, in table (14) 
they retain the positive sign. That is, the table is perfectly symmetri- 
cal. The method of obtaining the constant terms by summing rows 
is identical in both cases. 

Now let us consider the matrix of the coefficients of (21). An 
examination of these equations shows that the matrix is given by 


Auj=n-1 +1 +1---> +1 
+1 n-1 +1--- +1 


+1 +1 n—1--- 41 (23) 





ee | ee: cee 

A comparison of (19) and (23) shows that they differ only in that 
the 1’s in (19) are negative, while in (23) they are positive. In both 
matrices the diagonal terms are the same, viz., (n — 1). 


Now the determinant of (23) does not vanish, so that equations 
(21) determine uniquely the values of the S’s. However, in an experi- 











te 


PSE eR Se OG I le AR ae 


apie guilt 
CATT 


ie, ae 


on os i Sane 


SE eeEeEE—E— 


ee 


oe 


et cee 
4 wt aes 


M4 


430 The Journal of Educational Psychology 


mental set-up it is usually not feasible to get a matrix as indicated in 
(23). This is due to the fact that for some situations the proportion 
of judgments S; + S; > 0 is close to either zero or unity and hence 
too unreliable to use. For the same reason the matrix (19) obtained 
from experimental data differs also from that given. In an experi- 
mental set-up many of the elements which are —1 and 1 in (19) and 
(23) respectively are zero. When this is true the diagonal terms 
will not be (n — 1), but for any row the diagonal term will be equal 
to the number of 1’sin that row. If it were possible to obtain complete 
equations of the types (18) and (21) respectively, then the K’th equa- 
tion of (18) could be added to the K’th equation of (21) giving simply 


K-1 n 
fam ee mea (2 > bx-i - be) + Bes (24) 


K+1 


so that Sx could be obtained directly by adding the summation of the 
K’th column in (12) to the summation of the K’th column in (17) and 


/2 n(n — 1) 


multiplying this sum by pe Vy If we have — observation 
equations of the type S; — S; = b:_;~/2 and the same number of 
the type S; + S; = bi4;+/2 then (24) is the expression which makes 
>: + 8; — b;.;\/2)? a minimum where the sign + indicates that 
the summation is through both the S; — S; and the S; + S; series. 
As a matter of fact, in an experimental set-up for equations of 


the type S;+ S; = bi, 4/2 2 the b’s can be obtained only for S; + S; 
when 


S;>0> 8; 
or 
S; <0<S; (25) 


The inequalities (25) may not be required providing the ratio 
S;/o; is comparatively small. If these do not obtain, then (23) takes 
the form 


Auwj=lan . 0 1 . 1 
10 a 1 1 

” 26 

1 1 Om+1 0 ( ) 
1 0  & 
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where the S’s corresponding to all terms up to (m) are negative and 
those from (m+ 1) to (n) are positive, but where the a’s are still 
equal to the sum of the 1’s in their respective rows. 

Now (26) vanishes, so that if the experimental data result in a 
determinant of this type there is no solution for the S’s in (21). 

We next note that when the matrices obtained from the experi- 
mental data vary from (19) and (23) respectively this variation is 
due to the fact that certain of the b’s in tables (12) and (14) are missing. 
But knowing which of these b’s are missing the matrices can always be 
set up directly from the tables. 

But to determine the S’s uniquely, equations (18) and (21) are 
added together yielding a matrix which is the sum of (19) and (23). 

The normal equations may be set up directly from tables (12) 
and (14) as follows: 

To get the diagonal element for any row, count the total number of 
entries for that row in the corresponding rows of both tables. This 
number will be the diagonalelement. To get any other element: 

(1) If its corresponding row and column has an entry for both 
(12) and (14) the element is zero. 

(2) If its corresponding row and column lacks an entry in both 
(12) and (14) the element is zero. 

(3) If its corresponding row and column has an entry in only 
table (12), the element is —1. 

(4) If its corresponding row and column has an entry in only table 
(14), the element is +1. 

In a numerical set up, only those terms above the diagonal need 
be written down. 

The constant term for any row is obtained by adding the sum of 
the elements in the corresponding column in table (12) to the sum 
of the elements in the corresponding column of table (14) and multi- 
plying this total by +/2. 

The equations may be solved most conveniently by the Doolittle 
method (4). The solution is considerably simplified since all of the 
coefficients except the diagonal terms are either plus or minus 1, or 
zero. 

3. Correspondence of Calculated with Observed Values.—To find 
the degree of accuracy with which the solutions fit the experimental 
data, we determine the average squared deviation in the equations 


(S; = S; — b;_~/ 2)? = d?;_; 
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and 
(Si + 8; — bi V/2)? = dig; 


where the values obtained for the S’s are substituted in the equations. 

The value d,_; is the discrepancy obtained by subtracting the 
observed difference between the scale values of two stimulus situations 
from the calculated difference. The value d,,; has the same meaning 
for the sum of two scale values. The square root of the average of 
all of these deviations is called the standard error of estimate and is 


written 
yd? 
Cest. = — 
: n 


where (n) is the total number of both sum and difference equations. 
Of course, ¢,,, may be reduced to PE, by the relation 


PE,,,. = 0.6745¢,,, 
Now in any regression equation of the general form 
Y = By, + Bote +--+ + + Barn 


the PE,,, means that approximately fifty per cent of the experimental 
values will not deviate from the corresponding calculated values by 
more than the numerical value of the PE,,,. It may be interpreted as a 
measure of the error in the prediction or regression equation. 

But in our problem the regression equation takes the simpler form 


S; -_ S; = b;_;/2 
or 
Si HSi=[bu v2 


Here, then, the PE is a measure of the error in a sum or difference of 
two values. But the error of a sum or difference of two values is equal 
to the square root of the sum of the squares of their respective errors. 

Hence, if PE,,, = PE;; is the error in S; — S; or S; + S; and PE 
is the error in any given scale value, then 


PE,; = V (PE;)? + (PE;)? (27) 


If we assume that the errors in all the scale values are equal we obtain 
from (27) 





PE; = | (28) 
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To get PE;; we need not actually find the residual for each equation 
and then square and sum. From the method of fitting it follows that 
for any ‘‘least square”’ solution of the type 





D(Bix; + Bote +--+ Bat, — Xo)? = a minimum, 


that the standard error of prediction (5) is given by, 





a 2s — (Birox1 + Borot2 + - - - Berets) (29) 


n 


It will be noted that =z»? is the sum of the squares of the dependent 
variable. In our problem this is 


LZ" = 22b?,..; (30) 


where the sign + indicates summation through both the difference 
and sum series. Furthermore, 


B, = Si, B. = S2,°-:- Ba = Sn (31) 
and 


L221 = Ki, LLoX2 = Ko, es DE TN = ; (32) 
Substituting values (30), (31), and (32) in (29), we have 
_ 220% ~ 28K)" 


Cet. = 
n 





(33) 
or from (28) 





PE, = 0.6745, [720% = 2SiKi 


Equation (32) is the expression for the PE of a scale value. 

To obtain the value 2b?;;, tables are made up from tables (12) 
and (14) by squaring the values in these tables. Only the sections 
above the diagonals are included in the tables of squares, as those 
below have the same value when squared. The sums of the rows 
of both tables are added together to get 2b?;;. 

The value 2S;K; is readily obtained by multiplying each constant 
term by the corresponding scale value and summing. 





(34) 


IV. EXPERIMENTAL 


1. Method of Obtaining Values for the Difference and Sum Equations. 
The equations to be kept in mind in an experimental set-up are 
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2i_;-(Si_S}) 
Py5 = | f(zs-1) dav; (7) 
with its solution for S; — S;, 
S; — 8; = bi-in/2 (9) 
and 
2i+j-(Si+ Sj) 
Pigg = |S (ris) dri4; (10) 
with its solution 7 
Si + S; = bi/24; (11) 


The method for determining P in (7) is as follows: 
A series of stimulus situations is given ranging in affective value 
from distinctly positive to distinctly negative. Denoting the situa- 


Sn Sp- Sz S, 


i i i A i 4 


U ra) p 


Fia. 7. 





tions by Si, Sze to S,, we may represent them diagrammatically on a 
scale as in Fig. 7: 

Now when we say that situation 1 is more pleasant than situation 2 
for a given individual, we mean that he prefers 1 to 2. By definition, 
1 has for him a higher affective value than 2. Hence, to find whether 
situation 7 is greater in affect than situation j, statements of the 
situations are assembled according to the method of paired comparisons 
on mimeographed sheets with phrasing as follows: 


( ) 1-2. I would rather have S, than S:. 
( ) 1-3. I would rather have S, than S83. 
( ) 1I-n. I would rather have S, than S,. 
( ) 2-3. | I would rather have S;2 than S;. 
( ) 2n. I would rather have S, than Sy. 


( ) (n-1)-—n. I would rather have S,_; then Sn. 


These are presented to a group of individuals. If the individual 
considers the statement true for himself, he marks (+) in the paren- 
thesis. If not, he marks (—), just as in a True-False test. He is 
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instructed to leave no blanks. For each statement of paired situa- 
tions, then, there is a certain proportion of individuals who marked 
it (+). This is the value of P in (7) and from which the value };_; 
in (9) is obtained from a table of the normal probability integral. 
These b;_; values are the ones entered in table (12). 


When we say that situation 1 is more pleasant for a given individual 


than (n) is unpleasant, we mean that if he were permitted to enjoy 1, 
he would be willing to submit to (n). By definition, the cumulative 
affect of the two situations is, for him, positive. Hence, to find how 
much the cumulative affect of situations 7 and j differ from zero, the 
situations are assembled on a second set of mimeographed sheets 


with phrasing as follows: 


( ) 1-2. If I could have S,; I would be willing to take S:. 
( ) 1-3. If I could have S, I would be willing to take S;. 
( )1—mn. If I could have S,; I would be willing to take S,. 
( )2-3 If I could have S2 I would be willing to take Ss. 


( )2—n. If I could have S2 I would be willing to take S,. 
( ) (n-1)-n. If I could have S,_; I would be willing to take S,. 

These are presented to the same group of individuals as was the 
first series. Again for each statement of paired situations, the pro- 
portion of individuals who marked it (+) isobtained. This percentage 
is the value of P,,; in (10) and from which the value };,; in (11) is 
obtained from a normal probability table. These b,,; values are the 
ones entered in table (14). 

On the mimeographed forms for the first series, however, pairs of 
situations which are obviously far apart on the scale may be omitted, 
for they would result in proportions either too high or low to be reliable. 
For the same reason in the second series, situations which are obviously 
close together, and clearly some distance from the point of zero desir- 
ability or affect, are also omitted from the series. 

In the experimental series the situations were obtained from a 
number of the individuals who judged the pairs, as follows: 


The subjects were one hundred sixty students in a teachers college. 
The following instructions were given: 


Describe in one sentence a situation which you consider extremely unpleasant, 
one which you consider extremely pleasant, one which you consider as being 
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4 

i: entirely neutral, another which is moderately unpleasant, and still another which q 
i is moderately pleasant. t 
ae 

" Thus each individual submitted five situations. In all, about two t 
‘Ez hundred situations were obtained. The experimenter then selected 

if twelve out of these two hundred according to two criteria: 

te (1) Subjectively they appeared to be relatively uniformly extended b 
* through a scale from extreme unpleasantness to extreme pleasantness. \ 
ie (2) When possible, situations were selected which were most fre- 

“ quently described. 7 
e The object of (2) was, of course, to select situations which were 

i relevant to the everyday experience of the group which was to judge g 


them. They were then edited so as to fit as logically and gram- 
matically as possible into the form, 
I would rather have S; than S;, 
and T 
I would be willing to take S; if I could have S;, respectively. 
The situations which were put in the mimeographed forms were 
phrased as follows: 


tee 


ne 


Cee 


Receive a kiss from the one I love. 

Dance with a good dancer. 

Go to a good musical comedy. 

See a beautiful art exhibit. 

Receive a smile from a friend of the opposite sex. 

Watch a strange man try to choose between a red and a green necktie. 
Lose several hairs from my_ head. 

Hear a beginner practise his violin lesson. 

Discover I had forgotten my purse after having purchased an article. 
10. Have my clothes splashed by a passing vehicle when going to a party. 
11. See my sweetheart come into the show with another date. 

12. Spend a night in jail. 


PPMP Pr HPP 


2. Data and Calculations.—Table I gives the (b) values for both 
eis the sum and the difference series. Table II shows the normal equa- 
is tions set up from Table I in the manner previously outlined, and ready 
i for solution. The scale|values, as determined from a solution of these 
equations is given in Table III. 

3 To find the PE of a scale value, we employ the formula 
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PE = 0,67454|7>" i pees (32) T 
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To get 2b?,; another table is constructed from Table I by squaring only 


those b’s above the diagonal. 


by 2, we get, 


We find that 


22b?;; = 150.6480 


The value 2S;K; is obtained by multiplying column K of Table II 
by the column of scale values in Table III and summing the products. 


2S:K; = 145.5676 
The number of observation equations is 
N = 56. 


Substituting in the formula 


150.6480 — 145.5676 
112 


The PE of a scale value, then, is 0.1442. 


PE = 0.6745 








TaB.Le I.—SicmMa VALUES OF PROPORTIONS 


= 0.1442 


Multiplying the sum of all the b”s 











Si Ss 8:3 Ss Ss Ss Sr Ss Ss Siw Su Sis z= 
Series 2 

a Se ee Ie ee eee ee eee ee — .34/— .78|— .99|— 1.51/—3.62 

a one Sey ae ey Se es ee 47|— .61|/—1.19]|— .98]— 2.01] —4.32 

a eee SS Se Ae Seek Ne ey 1.39}— .21/— .94/— .79|— 1.96/—2.51 

in, Set See Pe ae ee ee — .45) .87/— .61|—1.10)—1.08)....... —1.47 

a Sean Cee ones ee SR See — .45|— .05|—1.23)—1.55|—1.67]....... —4.95 

rn Se Se ee ee ee — GIl— .20—1.77)......]...ccc)ecseees —3.38 

0 Be 45|— .45/— .91 

S Wo .47| 1.39} .87/— .05|— .70 

9 — .34/— .61/— .21|— .61]—1.23}—1.77 

10 =- —1.19}— .94|—1. 10) —1.55 

11 — .99|— .98|— .79|—1.08|—-1.67 

12 —1.51|—2.01]—1.96 

Series 1 

a — .58\— .04/— .39 

2 [ » -38 06] - .66 

3 04)— .38]...... — .50|—1.26|—1.57 

4 .30|— .06} .50)...... — .85|—1.38]—1.85 

i = 66] 1.26] .85)...... —1.07|—1.85 

a. a oe: 1.57} 1.38] 1.07]...... —1.63]—1.28]~—1.70 

Wi cclcccaee 1.85] 1.85] 1.63}...... 44|— .63|}—1.17 

i Ser ee ee ee 1.28|\— .44]...... —1.35|—1.90| —1.54 

i ES Oe Te ee eee 1.70} .63} 1.35)...... — .58|— .77|— 2.00 

— a SO: Sa ee Te Sey 1.17} 1.90} .58)...... — .42|— 1.63 

a er a ee Ry ee ee eee 1.54 .77| .42)...... 1.11 

Si ery SE nie! Cee ee SREB Oe 2.00} 1.63) 1.11 
ee —2.61|—4.68] 1.16} 1.78|—4.80|—2.79|—4.88| 5.93]/—5.10)—7.16| —7.13]—10.22 
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TaBLeE III 


SITUATION ABSOLUTE ScAaLE VALUES 


2.49 
2.48 
1.93 
1.90 
1.03 
0.17 

—1.01 

—1.55 

—2.64 

—3.39 

—3.57 

12 —5.15 


V. SUMMARY 


1. The scaling method developed by Thurstone is described, and 
the assumptions underlying the method are presented. 

2. The equations required in solving for the absolute affective 
values of a series of stimulus situations are derived analytically. The 
derivations include the assumptions involved in the method of scaling. 

3. Two methods for determining scale values are presented: 

(a) The “average difference’ method yields only scale separations 
and necessitates the arbitrary selection of an origin. 

(b) The ‘‘least square’? method gives a solution of the affective 
values of stimulus situations in terms of their distances from an 
absolute zero. 

4. The normal equations for the least square solution are derived 
and their typical form discussed. 

5. Instructions are outlined for writing the normal equations 
directly from the table of sigma values of the experimental 
proportions. 
~——-6. Statistical procedure is presented for determining how accurately 
the calculated scale values fit the experimental data. 

7. An experimental method is devised for obtaining data in the 
form required for the scaling equations. 

8. To demonstrate the scaling method a model experiment was 
conducted; the data resulting from the experiment are analyzed accord- 


ing to the method outlined and the results of the calculations are 
presented. 
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VISUAL IMAGERY AND “OBSERVATION” 


HENRY BOWERS 


Ottawa Normal School 


It has been pointed out! that certain stimuli evoking visual, audi- 
tory, or kinaesthetic images fall into a relatively constant order accord- 
ing to the mean ratings assigned by different groups to the aroused 
images. This suggested the possibility that the mean ratings assigned 
by groups to images evoked by the various items in a picture would 
show a similar relatively constant order of precedence. If this were 
so it would be interesting to compare this order with that into which 
the items fall when considered from the standpoints (a) frequency of 
mention in ‘‘reports,’’ (6) mean ranks in ‘‘reports.’’? 

Problems.—1. To determine if there is a relatively constant order 
of precedence for the items in a picture on the basis of clarity of the 
visual imagery that can be aroused by these items. 

2. If this relatively constant order of precedence exists to compare 
it with those previously established for frequency of mention of items 
and for mean order of mention in ‘‘reports.”’ 

3. If the order of precedence mentioned in (1) above exists, and if 
significant correlations are found between this order and those referred 
to in (2) above to determine if the relationship is a causal one. 

Procedure—Problem (1).—The subjects used were first and second 
year high school pupils and the test material was a coloured picture 
called ‘‘A Mountain River,” one in flat tones of a Labrador fiord, and 
Test 6 from the writer’s ‘‘Transfer Values of Secondary School Sci- 
ence.’ The latter test consists of thirteen exercises contained in a 
booklet. On one side of each page is a geometrical or nonsense figure; 
the back of the page contains a figure which could be made identical 
with the preceding one by the addition of certain details. It is the 





1 Bowers, H.: Constancy of Imaginal Content. Journal of Educational Psy- 
chology, Vol. XX, 1929, pp. 295-298. 

‘———_: Memory and Mental Imagery: An Experimental Study. Brit. J. 
Psychol., Vol. XXI, 1931, pp. 271-282. 

—: Studies in Visual Imagery. American Journal of Psychology, Vol. 

XLITI, 1931, pp. 216-229. 

* Bowers, H.: Order and Frequency of Observations Recorded by a Group. 
Journal of Educational Psychology, Vol. XXIII, 1932, pp. 221-225. 

’ Bowers, H.: ‘‘Transfer Values of Secondary School Science.”” The Univer- 
sity of Toronto Press, Toronto, 1927. 
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subject’s task to supply the details after examination of the first 
figure. Ten minutes is allowed for the thirteen exercises. Scores on 
the test made by six hundred twenty-one third- and fourth-year high 
school pupils are distributed in a close approximation to the normal 
probability curve. The mean reliability coefficient for a single grade 
is 50 + .07. Inscoring the test one point is allowed for each correctly 
completed exercise. That this procedure is legitimate may be inferred 
from the correlation coefficient .95 + .01 found to exist between ‘‘sum 
of correct elements’’ and ‘“‘sum of scores for correct elements.”” The 
latter scores were obtained by weighting each element for difficulty 
in the usual manner. 

‘“‘Reports”’ for each of the two pictures were obtained as described 
in ‘‘Order and Frequency of Observations Recorded by a Group.’ It 
may be mentioned that two minutes were allowed for examination of 
the picture and at the end of this period the subjects were required to 
write everything that they could recall about it. For present purposes 
only substantives were scored; attributives were ignored. After these 
statements were scored a list was prepared of the recorded items. 
Each subject was supplied with a copy of this list and a copy of the 
picture. He was requested to take each item in turn, find it in the 
picture, and examine it closely. Then he was to close his eyes and 
evoke the image. This image was rated on a four-step scale. Values 
0, 1, 2, and 3 were arbitrarily assigned to the four stages. Each 
subject had had previous experience in rating visual images. 

After scoring Test 6 the subjects were requested to rate the images 
aroused by each of the completed figures. 

The mean rating for each item in both pictures and for each ele- 
ment in Test 6 was found for random halves of the groups tested. 
Reliability coefficients for these mean ratings are given in Table I. 


TasBLE I.—REwIABILITY COEFFICIENTS FOR MEAN RaTINGs oF IMAGES 





No. of subjects 


‘ No. of items 
in each group 


Test material 





“Mountain River”’....../......... 24 and 24 








| 19 84 + .04 
‘“‘Tabrador Fiord”’........|......... 24 and 24 | 16 69 + .09 
| 13 91 + .03 


REL GR cae ae Ee A 24 and 24 





These coefficients extend to the constituent elements of pictures 
the property of relative constancy of imaginal content shown to exist 
for other material. It is obvious that the mean estimate for the whole 
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group of forty-eight is much more stable than those for the two sub- 
groups of twenty-four. 

It is interesting to compare this property of the elements composing 
a picture with E. R. Jaensch’s statements concerning the suitability 
or unsuitability of certain pictures for the discovery and investigation 
of eidetic imagery. Hesays'.. . ‘the appearance of the conceptual 
factor is not so dependent on fixation or on factors that influence the 
sense organ. It depends rather on the interest taken in the picture 
and on the involuntary attention aroused by the picture . . . Certain 
types of objects are preferred and sometimes give rise to clearer EJs? 
by themselves quite independently of the purely optical qualities and 
particularly of the sensory intensity of the picture. Animal pictures 
pick out a higher percentage of EJs from children.’’ In another place 
he criticizes Schwab’s® use of a test picture as “‘it fails to conform to 
almost every condition for a satisfactory test-object. It lacks (1) 
a sensible coherence; (2) it contains a large number of similar or 
approximately similar elements; (3) there are a number of regions of 
the same colour and shape sometimes symmetrically arranged.”’ 

Problem (2).—The groups which provided ‘reports’? on both 
pictures were divided into halves by random selection. For each half 
the mean frequency and mean rank were calculated for items appear- 
ing in these ‘‘reports.’’ Reliability coefficients for these mean 
frequencies and mean ranks are reported elsewhere (‘‘Order and Fre- 
quency of Observations Recorded by a Group”). The mean r for 
frequency was .85, that for rank, .70. Table II shows the relationship 
found between imagery rating, mean frequency, and mean rank. 
TaBLE II.—CorRRELATION BETWEEN MEAN RaTING OF ITEM FOR IMAGERY AND 


MEAN FREQUENCY IN REPORT. CORRELATION BETWEEN MEAN RATING 
or IreM FOR IMAGERY AND MEAN RANK IN REPORT 








Picture No. of No. of eenulis and sien and 
subjects items f \ k 
requency ) rank) 
“Mountain River’’....... 48 19 +.59 + .10} —.75 + .07 
“Labrador Fiord”’........ 28 16 +.36 + .15 
Labrador Fiord........... 28 ma 6 OD ueeadaneeda —.48 + .16 




















1 Jaensch, E. R.: ‘‘Eidetic Imagery.”’ Kegan, Paul, Trench, Trubner & Co., 
1930. 

2 Kidetic images. 

?Schwab, G.: Vorlaiufige Mitteilung tiber Untersuchungen zum Wesen der 
subjektiven Anschauungsbilder. Psychol. Forschung, Vol. V, 1924. 
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The coefficient of correlation between the mean score for each of 
the thirteen elements in Test 6 and the mean imagery rating for each 
of these exercises was +.62 + .12 (forty subjects). Another deter- 
mination gave r= +.57+ .13 (fifty subjects). Each of these 
coefficients was obtained from a single grade. It is estimated that 
correction for attenuation should increase all of these rs by approxi- 
mately one-fifth. 

It is apparent that there is an association between mean frequency 
of mention in ‘‘reports’”’ and mean rating for aroused imagery. There 
is also a tendency for a group to record those items in “‘reports”’ in an 
order approximating to that in which they were rated for their effective- 
ness in evoking visual images. It would also appear that there is a 
tendency for those exercises in Test 6 which lent themselves most 
readily to reproduction by visual imagery to be solved most easily. 

Problem (3).—Although significant correlation exists between the 
facility with which the component items of a picture are reproduced in 
visual images and the frequency and rank of these items in ‘‘reports”’ 
it cannot be concluded that clarity of imagery determines even partially 
this rank and frequency. Further data are necessary. 

The writer’s test of visual imagery (‘‘Studies in Visual Imagery’’) 
was given to thirty-seven subjects from whom ‘‘reports”’ on ‘“‘ Moun- 
tain River” had been obtained. Total scores on the visual imagery 
test and total scores on the “‘reports” gave r = +.22 + .11. The 
correction for attenuation would raise this coefficient to approximately 
.25. (7, for the test in visual imagery is .96 for a single grade, that 
for the ‘“‘report”’ is certainly not above .8). Two values were found 
of the coefficient of correlation between scores on Test 6 and scores on 
the test for visual imagery. They are +.15 + .10 (N = 40) and 
+.05 + .12 (N = 32). 

No correspondence can be detected between the ability of a subject 
to evoke clear visual images of pictorial representations and the ability 
to remember these and record them. In view of the established 
relationship between frequency and rank (‘‘Order and Frequency of 
Observations Recorded by a Group”’) it may be assumed that the 
relationship between imagery and order is not causal. 

Remarks on Method.—1. An apparent limitation of our technique 
is that although only two minutes was allowed for examination of the 
picture before a ‘‘report’”’ was required, as much time as desired was 
given when rating the clarity of mental imagery aroused by each item. 
In defence of this is the r of +.86 + .06 obtained between total scores 
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on “reports” in which a two-minute observation period was allowed 
and those secured when using a five-minute period. Eighty-three 
days separated the two sets of reports. 

2. It is unfortunate that more items were not recorded. This is 
due of course to the nature of the pictures and also to the limitations 
of memory. The calculation of mean rank in the case of several items 
was impossible in view of the extreme unreliability involved in the use 
of very few cases. These items were discarded. 

3. A certain degree of subjectivity is involved in scoring the reports. 

However, in view of the fairly consistent coefficients obtained it 
would seem that in spite of the difficulties referred to above, our con- 
servative conclusions have adequate basis. 

The matter of subjectivity in the test for visual imagery has been 
discussed elsewhere. 


CONCLUSIONS 


1. The constituent elements of a picture exhibit that relative 
constancy of imaginal content found to exist for other visual, auditory, 
and kinaesthetic stimuli. 

2. A small but significant correlation exists between mean ratings 
of images aroused by items in a picture and frequency of mention of 
these items in ‘‘reports”’ of the picture. 

3. A rather more pronounced association exists between clarity 
of image and mean rank of item. 

4. There is evidence that the relationships referred to in (2) and 
(3) are not of a causal] nature. 

Evidence supporting conclusions (1), (2) and (4) was obtained 
from a test involving the completion of geometrical and nonsense 
figures. 





1 Bowers, H.: Studies in Visual Imagery. Am. J. Psychol., Vol. XLIII, 1931, 
pp. 271-282. 











“STUDENT'S” METHOD FOR MEASURING THE 
SIGNIFICANCE OF A DIFFERENCE BETWEEN 
MATCHED GROUPS 


MORDECAI EZEKIEL 
Federal Farm Board 


In a recent article,! Lindquist shows that when the effectiveness of 
two methods of teaching are compared by the use of ‘‘ matched groups,”’ 
the correct probable error of the difference between the mean of two 
groups may really be very much smaller than the figure computed by 
the usual formula for the error in the difference of means. He further 
explains a new formula, developed by Wilks,” the ‘‘standard error of 
the mean of matched samples,’’ which serves as a means of estimating 
the increased significance of the results obtained by the ‘‘matching” 
process. 

As is not unusual in the progress of science, the development of 
this method by Wilks and Lindquist represents a rediscovery, by 
workers in one field, of what has long been known by workers in 
another. Over twenty years ago, an eminent English statistician 
“Student” (he writes under this pseudonym) considered the same 
type of problem which Lindquist attacks, and arrived at parallel 
conclusions.* As it happens, however ‘‘Student”’ developed a method 
of measuring the accuracy of the mean difference between the two 
matched groups which requires much less work to compute than does 
the Wilks-Lindquist formula. With the thought that psychologists 
and educators might be able to save much energy by the use of this 
simpler method, this article is prepared to call their attention to its 
existence. 

The method proposed by Lindquist will be illustrated by a set of 
numerical data, and then ‘‘Student’s’”’ method will be shown by 
applying it to the same data. 





1 Lindquist, E. F.: The Significance of a Difference between ‘“Matched”’ 
Groups. Journal of Educational Psychology, Vol. X XII, 1931, pp. 197-204. 

2 Wilks, Samuel S.: The Standard Error of the Means of ‘‘ Matched”’ Samples. 
Journal of Educational Psychology, Vol. XXII, 1931, pp. 205-208. 

3 Fisher, R. A.: ‘‘Statistical Methods for Research Workers.’”’ Second edition, 
1928, pp. 104-122. 

‘‘Student”’: The Probable Error of a Mean. Biometrika, Vol. VI, 1908, pp. 

1-25. 
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We will use the same example that Lindquist considers, an experi- 
ment to determine the relative effectiveness of two methods of teaching 
subtraction, with one group of children taught by the ‘‘additive”’ 
method, and another by the “‘take-away”’ method. After a period of 
instruction, a final measure of ability in subtraction would be obtained 
for all the pupils in both groups. To reduce random errors in the 
experiment, the two groups would be “‘matched” by trying to have 
the average initial ability of the pupils the same in each group. ‘‘If 
the intelligence test scores had been used, the two groups would have 
been so selected as to have identical distributions of intelligence test 
scores; in other words, they would have been ‘matched’ on the basis 
of intelligence.’’! 

The following set of hypothetical data illustrate the results that 
might be secured by an experiment using the ‘“‘matching”’ method. 
(Only ten cases are shown in each sample; in practice, larger groups 
would ordinarily be required for dependable results.) 


TABLE I.—HYpoTrsHeETIcAL DaTa ILLUSTRATING RESULTS SECURED BY ‘‘ MATCHING”’ 

















PROCESS 
Group A Group B 
sa Final rating in | ail | Final rating in 
pues SN subtraction man oS | subtraction 
| | 
80 58 75 65 
90 63 88 67 
95 | 83 98 85 
98 | 78 99 83 
103 80 103 86 
105 88 107 88 
110 85 108 90 
122 86 119 89 
135 91 138 96 
160 94 155 98 
Averages... .109.8 80.6 109.0 84.7 














The data for the children in both groups are arranged in order of 
increasing IQ, from lowest to highest. In addition to the average IQ 
being almost exactly the same for both groups, the children have been 
so selected that the distribution is nearly identical in the two groups, 





1 Lindquist: Loc. cit. 
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for each child in the first group there being a corresponding child in 
the second group, with about the same IQ. 

The average rating on subtraction for Group A is 80.6, and for 
Group B is 84.7. Is the difference of 4.1 points significant evidence 
that the method used in teaching Group B is superior to the method 
used in teaching Group A? 

By the usual method, the probable error of the difference in the 
averages between two groups is given by the formula:! 

Cg ;? 02 

PE)_2 == 67454) — oe + je <= 1 

(this differs from the formula frequently given by substituting n — 1 
for n.) 


When the number of cases in both groups is the same, (as in this 
case) the formula becomes 


PE}-_2 = V/V PE}? + PE;? 


= 6745 athe Sn 
. — = l 


For the example shown, the standard deviation of the subtraction 
scores (cs) is 11.07 for Group A, and 10.26 for Group B. Substituting 
these values in the last equation, the probable error of the difference 
between the two averages is found to be 


PE... = 674s, [LO + (1026) 
s=. ; 





2 














= 3.39 


Using this method, the difference between the scores of the two groups 
would be stated to be 4.1 + 3.39, which would be taken to mean that 
the difference observed could readily have occurred by chance and that 
therefore there was no significant difference between the results secured 
by the two methods of teaching. 

The Wilks-Lindquist formula recognizes the correlation between 
the criterion used in ‘‘matching’”’ the groups, and the final scores. 
Using I for IQ, and S for score, (and substituting n — 1 for n) the 








formula is 
ov 1 — rig 
Ow = 
" a/n —]1 


where cy: represents the “standard error of means of matched groups. ” 





1 Fisher, R. A.: ‘Statistical Methods for Research Workers.” P.109. (This 
is simplified from the form as Fisher gives it.) 
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For the standard error of the difference between two matched 
groups, the formula is 


anil nS =1* =a) i 


For the twenty cases in Groups A and B the correlation between IQ 
andsubtraction score is 0.813. Substituting this value in the last equa- 
tion, the probable error of the difference is: 


PE: = e745) O97 + =| (1 — (0.813)"] 











9 9 
= 1.98 


Hence by the Wilks-Lindquist method the difference between the 
scores of the two groups would be 4.1 + 2.0. While a difference as 
large as this might still occur by chance, this would indicate a good 
probability that the method for Group B was superior to that for 
group A, whereas the usual method indicated that there was a con- 
siderable probability that the difference was due solely to chance. 

“Student’s” method is based upon an entirely different approach. 
It compares the scores for each child in Group A with the score of the 
corresponding child in Group B, and asks whether the average of these 
differences in score is significant when contrasted with the variation in 
the individual differences. It is illustrated in Table IT. 


TaBLeE II.—CompuTaTION OF DIFFERENCE IN RATING IN SUBTRACTION FOR 
‘“MatTcHED’’ CHILDREN 

















Final rating in subtraction oe 
IQ rating (B — A) 
Child in Group A | Child in Group B 
75 and 80 58 65 7 
88 and 90 63 67 4 
95 and 98 83 85 2 
98 and 99 78 83 5 
103 80 86 6 
105 and 107 88 88 0 
108 and 110 85 90 5 
119 and 122 86 89 3 
135 and 138 91 96 5 
155 and 160 94 98 4 
BR 5 o's 46 &%0 nae 80.6 84.7 4.1 
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All that remains to be done is to calculate the standard deviation 
of the individual differences, (B — A). This comes out 1.90. The 
probable error of the differences may then be calculated by modifying 
the formula for the probable error of a mean: 


2 
PE;_4 = 67454] ee 








n—1 
Which in this case gives the value 





in 
2 
PEs-4 = .6745 a 

= .43 be 

So using “Student’s’”’ method, the difference in average score 
between the two matched groups would be stated to be 4.1 + 0.43, be 

indicating that method B is superior to method A, and that it is almost 
a certainty that the difference could not have occurred as a result of de 
chance fluctuations, even with samples as small as ten.' an 
“‘Student’s’’ method can be applied to matched samples only when pu 
they have been so matched that for each case in one sample there is a cli 

corresponding case in the other sample, with practically identical 
characteristics insofar as the ‘‘matching”’ criterion is concerned. Thus ye 
in the arithmetic example shown, the two IQ’s were almost identical we 
for each pair of corresponding children. If the general distribution of of 
the criterion was the same in each sample, but the case-for-case aid 

correspondence did not hold, ‘‘Student’s”’ method could not logically be 
applied. of 
It is evident that where it can be applied, ‘“Student’s’’ method 2 


requires much less computation than does the Wilks-Lindquist method. , 
The latter method requires the computation of two standard deviations BIN 
and one Pearsonian coefficient of correlation, whereas the former 





requires the computation of the differences shown in Table II, and of elg 
their standard deviation. Furthermore, ‘“Student’s’’ method gives a tw 
more exact measure of the reliability of the differences. While the age 
Wilks-Lindquist formula gives a more accurate estimate of the relia- dif 
bility than does the ordinary formula, it still much under-estimates the me 
significance of the results, as judged by ‘‘Student’s” method. F 
1 According to the tables for small samples, a departure as large as this (with to 
the average over six times its standard error) would occur by chance less than the 
once out of one hundred samples. See Fisher, R. A.: ‘‘Statistical Methods for 
Research Workers,”’ p. 139; Mordecai Ezekiel, ‘‘ Methods of Correlation Analysis,” old 


Table A, 1930, p. 20. log 
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STANDARDIZATION OF THE STENQUIST 
MECHANICAL ASSEMBLYING TEST 
SERIES III 


EDNA WILLIS McELWEE 


In addition to standardizing the Stenquist Mechanical Assembly- 
ing Test Series III, the purpose of this study was three-fold: 

1. To correlate mechanical ability and intelligence. 

2. To compare the mechanical performance of dull and bright 
boys of the same chronological age. 

3. To compare the mechanical performance of older and younger 
boys of the same mental age. 

The Stenquist Mechanical Assemblying Test Series III was 
designed for younger boys. It consists of a plain bolt and nut, a bolt 
and wing nut, a plain hinge, a key and ring, a turn buckle, a drawer 
pull, a trunk caster, a plain push button, a belt and buckle and a nail 
clip. 

The test was given to four hundred boys six, seven, eight and nine 
years of age. There were one hundred boys in each age group. They 
were enrolled in Grades IA to VB inclusive. As practically every boy 
of these ages in the school was included in the study, it may be con- 
sidered a normal group. 

The boys were examined in groups of three, because that number 
of test boxes was available. A maximum time of twenty minutes was 
allowed to complete the test, but only a few boys spent that long. 
Many of them reached their limit in ten minutes. No time bonus was 
given in scoring as is done in Series I and II. 

Every boy was able to do something, the lowest score being 
eighteen. None of the boys made a perfect score (one hundred 
twenty), the highest being one hundred five. Two boys received full 
credit and eighteen others partial credit for the nail clip, the most 
difficult article to be assembled. Therefore, the writer feels that the 
mechanical ability of all the boys was fully measured. 

In Table I the scores made by each age group are tabulated in 
deciles, thus standardizing the Stenquist Mechanical Assemblying 
Test Series III and presenting norms for scoring. Although some of 
the six year old boys make a higher score than some of the nine year 
old boys, there is a steady gain in mechanical performance as chrono- 
logical age increases. 


451 








——— SO ee Oe eee ee 2. ee 


452 


The Journal of Educational Psychology 


TaBLE I.—NoRMS FOR THE STENQUIST MECHANICAL ASSEMBLYING TEST SERIEs 
III Basep on Four Hunprep CaszEs 















































Decile Year 6 Year 7 Year 8 Year 9 
10 66-87 72-95 81-101 86-105 F 
9 56-65 65-71 72-80 81-85 re 
8 51-55 62-64 67-71 74-80 
7 47-50 57-61 63-66 70-73 V 
6 45-46 54-56 60-62 66-69 | 
5 41-44 50-53 56-59 62-65 of 
4 37-40 45-49 50-55 57-61 ty 
3 34-36 40-44 47-49 54-56 I\ 
2 28-33 36-39 40-46 50-53 C 
1 18-27 22-35 31-39 36-49 
Ts 
In connection with another study at the same school, group tests 
of intelligence had been given to a number of the classes. These IQ’s - 
were available for the boys seven, eight, and nine years of age. 
TaBLeE IJ].—IQ DisTRIBUTION FoR THE Boys SEVEN, E1igut AND NINE YEARS OF 
AGE _ 
Thi 
Number of boys Br 
Range of IQ —— —- — “ 
Year 7 Year 8 Year 9 
70-79 1 19 20 ch 
80-89 22 38 28 wi 
90-99 32 17 16 tr: 
100-109 23 14 16 
110-119 13 7 7 Se 
120-129 8 4 6 Fr 
130-139 0 0 6 
140-149 1 1 1 ye 
sel 
In Table II the IQ ‘distribution of these three age groups is pre- ; 
sented. By arbitrarily regarding the IQ range 70-79 as Borderline, wi 
80-89 Dull, 90-99 Normal, 100-119 Bright and 120-149 Superior, the ye: 
three hundred boys are grouped as follows: forty Borderline, eighty- ye 
eight Dull, sixty-five Normal, eighty Bright and twenty-seven Superior. an 
This grouping suggests a normal curve. 
Using the Product Moment formula, correlations were worked out no: 
between mechanical ability and intelligence for each of the three age = 
( 


groups. 
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Seven year group r= .102 PE + .067 
Eight year group r= —.206 PE + .064 
Nine year group ‘rf = —.058 PE + .064 


From the three correlations the inference is quite apparent that the 
relationship between mechanical ability and intelligence is negligible. 

Because the nine year age group was drawn from Classes IIIA to 
VB inclusive, it was possible to compare the mechanical performance 
of dull and bright boys of the same chronological age. There were 
twenty-six boys nine years of age in the bright sections of the Classes 
IVB, VA and VB and an equal number in the dull sections of the 
Classes IITA, IIIB and IVA. 


TaBLE III.—Comparison oF IQ aND MECHANICAL PERFORMANCE OF TWENTY-SIX 
BRIGHT AND TWENTyY-sIx Dutt Boys Nine Years or AGE 








Range Median 
Range of IQ | Median IQ| mechanical | mechanical 
score score 
Ee rT 73-109 82 38-105 74 
ae 100-142 121 45-101 65 

















Of course twenty-six is too small a group from which to draw con- 
clusions, but this comparison seems to bear out the belief that boys 
who do not excel in academic subjects would profit by industrial 
training. 

On another occasion the Stenquist Mechanical Assemblying Test 
Series III was given to a number of the older boys in ungraded classes. 
From this group sixty boys with mental ages of seven, eight and nine 
years were chosen. ‘T'wenty boys in each mental age group were 
selected. 

The IQ range of these boys was from fifty to seventy. The boys 
with mental ages of eight and nine years were all thirteen and fourteen 
years of age chronologically. In the group with mental ages of seven 
years, a few of the boys were eleven and twelve; the others thirteen 
and fourteen years of age. 

For comparison with the group of ungraded boys a group of sixty 
normal boys seven, eight and nine years of age chronologically was 
selected. There were also twenty boys in each age group. Only 
those with IQ’s between ninety and one hundred were chosen. 
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The scores made by these older boys of inferior intelligence in 
ungraded classes have been compared with those made by younger 
normal boys of the same mental age in Table IV. 


TaBLe IV.—ComMPARISON OF THE MECHANICAL SCORES OF Srxty UNGRADED Boys 
AND Srxty NorMAL Boys or THE SAME MENTAL AGE 





Range me- Average me- 
chanical score | chanical score 





re Ungraded 10-120 84 
Normal 36-101 64 
eer ree Ungraded 5-120 81 
Normal 35-100 55 
IB one ddan eccccesnan Ungraded 46-110 78 
Normal 25-72 51 














In each group the older boys made a much better showing in their 
mechanical performance than the younger boys of the same mental 
age. Could this difference be explained by social experience and 
interest? 


SUMMARY 


1. The Stenquist Mechanical Assemblying Test Series III was 
standardized for boys six to nine years of age. 

2. There is a definite relation between the mechanical performance 
and the chronological age. 

3. The correlation between mechanical ability and intelligence is 
negligible. 

4. Dull boys made a higher mechanical score than bright boys of 
the same chronological age. 

5. Older boys of inferior intelligence made a higher mechanical 
score than younger boys of the same mental age. 
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A MODIFICATION OF THE DOOLITTLE METHOD: A 
LOGARITHMIC SOLUTION 


R. J. WHERRY 


Cumberland University, Lebanon, Tennessee 


Much use has been made lately of the Doolittle method of solving 
normal equations, necessitated by the increased use of the multiple 
correlation technique. Its more extended use by Psychologists has 
been urged in a recent article in the JouRNAL oF EDUCATIONAL 
PsYCHOLOGY.? 

The purpose of this article is to present a modification of the method 
making it available, without so much multiplying involved, to those 
persons who do not have a calculating machine available. 

The method as derived by Doolittle is adequately set forth by 
Mills? and needs no elaboration here. Doolittle devised his procedure 
from the work of Gauss as set forth by Merriman in his text on Least 
Squares,’ and it is from this same source that the author has derived the 
present modification of the Doolittle method. 

The one great fault of the Doolittle method, as has been suggested 
above, is that it requires, for its satisfactory use, the availability of a 
calculating machine, due to the great amount of multiplying through- 
out the process. The present paper seeks to obviate this difficulty by 
substituting the use of logarithms instead, requiring only a five or six 
place set of logarithms. 

In order to illustrate the method let us consider the following 
example (the one used by Mills to illustrate the Doolittle method is 
used here for sake of comparison‘), whose normal equations are: 


8.3564bi0.34 + 2.790bi3.04 + 2.932b1403 + 47.967 = 0, 
+ 6.6645b13.94 — 2.063b14.03 _ 62.039 = 0, 
+ 7.7893b14.23 + 47.519 = 0. 


These equations are of course in the deleted form permitted by the 
method. 

Our problem is to find the values of the terms 6j2.34, 613.24, and 
bigeo3. All of the necessary work, with the necessary checks is shown 
in the accompanying table (Table I). 

Explanation of Table I—In the columns A, B, and C appear the 
coefficients of the “‘b’’ values, in column D the constant values, and in 
column E the sum of all of these previous values, on the basis of which 
the checking is done. 
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TasLE I.—So.vutTion oF NoRMAL EQUATIONS BY THE WHERRY-DOOLITTLE-GaAvss 
METHOD 
Li A B C D E 
_ Diz.ss bis.24 Dia.os (m) (s) 
I 8.3564 | 2.790 2.932 47 .967 62.0454 
a Peer): 6.6645 | 2.063 62.039 73.5565 
Sn) iMacs «ie actecudacuqwie 7.7893 | 47.519 60.3033 
1 8.3564 | 2.790 2.932 47 .967 62.0454 
2 . 92202 .44560 .46716 1.68094 1.79271 
3 .00000} 9.52358) 9.54514 . 75892 .87069 
4 —1.00000)/— .3339 |— .3509 |— 5.7401 — 7.4248(—7.4249) 
- Maceahcan 6.6645 | 2.0630 | 62.039 73.5565 
ee ee 9.96918) 9.99074 1.20452 1.31629 
ae err — .9315 |— .9789 |—16.0150 —20.7155 
a eee: 5.7330 | 1.0841 | 46.0240 52 .8410(52. 8411) 
Te eee . 75838 .03507 1.66299 1.72297 
Fa Sere .00000} 9.27669 .90461 .96459 
| -) eae —1.0000 |— .1891 |— 8.0280 — 9.2170(—9.2171) 
Bt ON Bik ds ex ciiendem wel 7.7893 | 47.519 60.3033 
7 eae oS eee .01230 1.22608 1.33785 
ae Serre Secrest —1.0288 |—16.8300 —21.7695 
— i eer ere 9.31176 . 93968 . 99966 
oa eee eee — .2050 |— 8.7032 — 9.9922 
7. eee Were 6.5555 21.9858 28 .5416(28. 5413) 
OE eee eee es .81661 1.34215 1.45548 
i, ee eee .00000 . 52554 .63887 
FF ae eee —1.0000 |— 3.3538 — 4.3539( —4.3538) 
—bis.23 + (—3.3538) = 0 
bis.es = — 3.3538 
—bis.2a + (—.1891)bi4.23 + (—8.0280) = 0 
| bis.2s = — 7.3938 
—bi2.se + (—.3339)bis.246 + (—.3509)b14.23 + (—5.7401) = 0 
bie.sa = — 2.0945 
Check: 
8.3564bi2.34 + 2.790b13.24 + 2.932b14.23 = —47.967 
8.3564( —2.0945) +!2.790( —7.3938) + 2.932(—3.3538) = —47.964 


In lines I, II, and III are inscribed the coefficients of the normal 


equations as indicated above. 


The steps numbered below correspond 


to the procedure followed in obtaining the results given on the corre- 
sponding line in the table. 
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1. The normal equation I is recopied in line 1. 

2. Write the logarithm of each number in line 1 in line 2. 

3. Subtract the logarithm found in column A, line 2 from each of 
logarithms in line 2, recording the result immediately below in line 3. 
Notice that in the table the log. 9.52358 is written in place of 
9.52358 — 10. 

4. Look up the anti-logs of the logarithms in line 3 and write in line 
4, giving them the sign opposite to the sign of the corresponding 
number in line 1 when the number in line 1, column A is plus, and the 
same sign as the numbers in line 1 when the number in line 1, column A 
is negative. (Check—the sum of the numbers in line 4, columns A to 
D, must equal the number in column £.) 

5. Copy normal equation II on this line. 

6. Add the logarithm appearing in column B, line 3 to each of the 
logarithms in line 2 (columns B to £) and record the results below in 
line 6. 

7. Look up the anti-logs of the logarithms in line 6 and record 
immediately below in line 7. Notice the sign of the number in column 
B, line 4 and multiply the signs of the numbers in line 1 by it and record 
the results before the corresponding numbers in line 7. 

8. Add algebraically the numbers in line 7 to those in line 5 and 
record the results in line 8. (Check—the sum of the numbers in line 8, 
columns B to D must equal the number in line 8, column £.) 

9. Look up and record the logarithm of each number in line 8 
immediately below in line 9. 

10. Subtract the logarithm in Jine 9, column B from each of the 
logarithms in line 9 and record the results in line 10. 

11. Look up the anti-logs of the logarithms in line 10 and record 
in line 11, giving them the sign opposite to the sign of the corresponding 
number in line 8 if the sign of the number in line 8, column B is plus, 
and the same sign as the number in line 8 if the sign of the number in 
line 8, column B is negative. (Check—the sum of the numbers in line 
11, columns B to D, must equal the number in line 11, column £.) 

12. Copy normal equation III on this line. 

13. Add the logarithm appearing in line 3, column C to each of the 
logarithms in line 2, columns C to £ and record the results in line 13. 

14. Look up the anti-logs of the logarithms in line 13 and record 
immediately below in line 14. Notice the sign of the number in line 4, 
column C and multiply the signs of the numbers in line 1 by it and 
record the results before the corresponding numbers in line 14. 
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15. Add the logarithm appearing in line 10, column C to each of the 
logarithms in line 9, columns C to E and record the results in line 15. 

16. Look up the anti-logs of the logarithms in line 15 and record 
immediately below in line 16. Notice the sign of the number in line 11, 
column C and multiply the signs of the numbers in line 8 by it and 
record the results before the corresponding numbers in line 16. 

17. Add algebraically the numbers in lines 12, 14, and 16 and 
record the result for each column in line 17. (Check—the sum of the 
numbers in line 17, columns C and D must equal the number in line 
17, column E.) 

18. Look up and record the logarithm of each number in line 17 
immediately below in line 18. 

19. Subtract the logarithm in line 18, column C from each of the 
logarithms in line 18 and record the results in line 19. 

20. Look up the anti-logs of each of the logarithms in line 19 and 
record in line 20, giving them the sign opposite to the sign of the 
corresponding number in line 18 if the sign of the number in line 18, 
column C is plus, and the same sign as the corresponding number in 
line 18 if the number in line 18, column C is negative. (Check—the 
sum of the numbers in line 20, columns C and D must be equal to the 
number in line 20, column £.) 

If there were additional unknowns, the process would be extended 
in exactly the same manner. 

The checks on the various operations have been indicated in the 
table and in the explanation. The testing of the results immediately 
following each step reduces the possibility of error to a minimum. 

The back solution for the ‘‘b”’ values presents no difficulty, being 
carried out as indicated in the lower part of Table I. 

The final check as to the accuracy of the work comes in the sub- 
stitution of the discovered ‘‘b” values in one of the complete normal 
equations. This is done for equation I in Table I. 


CONCLUSIONS 


1. The adaptation of Gauss’s method for the solution of normal 
equations as worked out by Doolittle, while it is very satisfactory for 
machine calculation, is excessively burdensome when such a machine 
is not available. 

2. The logarithmic solution presented in this paper is based upon 
exactly the same principles, and gives results of approximately the 
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same mathematical accuracy, and yet is suited to hand calculation. 
The author feels that this method will therefore be a boon to many 
workers in the smaller universities and colleges, whose budgets do not 
permit the purchase of expensive calculating machines. 
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NOMOGRAM FOR BLAKEMAN’S TEST FOR 
LINEARITY OF REGRESSION 


HAROLD D. GRIFFIN 


Nebraska State Teachers College, Wayne, Nebr. 


Blakeman has devised formulas! to be applied in testing a correla- 
tion ratio and coefficient of correlation to ascertain linearity or non- 
linearity of regression. There are many versions of Blakeman’s 
tests.2,3-45 The one used in constructing the accompanying nomo- 
gram follows Holzinger. For the theory the reader is referred to 
Blakeman’s original article. 

The best straight-edge to use in reading this and other nomograms 
is made by scratching a straight line with a needle on the underside 
of a strip of good weight celluloid. 





1 Blakeman, J.: On tests for linearity of regression in frequency distributions. 
Biometrika, Vol. IV, pp. 332-350 (1905). 

2 Blakeman: Op. cit. 

3’Garrett, H. E.: ‘Statistics in Psychology and Education,’ first edition. 
Longmans, Green and Co., New York, 1926, pages 209-211. 

4 Gavett, G. I.: ‘‘ First Course in Statistical Method,”’ first edition. McGraw- 
Hill Book Co., New York, 1925, p. 239. 

5 Holzinger, K. J.: “‘Statistical Methods for Students in Education,’ first 
edition. Ginn and Co., Boston, 1928, p. 183. 
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VN VE <4:047 


where [=7 27? and 
N= roth. Soc of cases. 


When the result is 
below 4-047, linearity 
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is indicated ; when abe -o80—= 
4-047, non-linearity. 070 -= 
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THE EFFECT OF PRACTICE ON THE HOMOGENEITY 
OF A GROUP 


ROBERT P. CARROLL 


Syracuse University 


When the subject of homogeneous grouping is discussed in classes in 
Education the question sometimes arises as to how long a group will 
remain homogeneous. As time goes on will the individuals in the 
group become more and more alike or more and more unlike in ability 
or skill? Still another question arises and that is, what measure best 
shows whether the homogeneity of a class or section is decreasing or 
increasing? In answer to this last question the mean shows nothing. 
The range has very little significance, for it may remain practically 
constant while the majority of the members of a group are getting 
farther apart or closer together in their achievement. The sigma is a 
very good measure, but it often increases as the average increases. 
Thus a class may actually be more alike in achievement after a few 
days of practice on a given function, while the sigma may be much 
larger than when the practice began. It would seem that the ratio 
between the mean deviation and the mean, or between the sigma and 
the mean, that is, the coefficient_of variation, would be the best measure 
of homogeneity. The latter measure was used in connection with two 
experiments that were conducted in Syracuse University during the 
last twelve months with groups that were comparatively homogeneous 
at the outset. 

The first experiment consisted of twenty-eight days of drill in one 
of the broader mental functions,! namely, that of silent reading. 
Changes in the C. V. are shown in Table I. 

The changes are by no means regular, nor do they continue con- 
sistently in any given direction. ‘The fluctuations are rather similar 
to those found in general in the results obtained from practice or drill 
experiments. For the most part, however, the trend is downward, 
which indicates an increase in homogeneity. This table represents 
the work of twenty-eight college students in the summer session of 
1929. | 

The second experiment was conducted with the cooperation of 
thirty-nine students in Educational Psychology in Teachers College. 





1 Drill in Silent Reading for College Students. Reported in School and Society, 
November 9, 1929. 
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TaBLE I.—SHowina CHANGES IN THE COEFFICIENT OF VARIATION IN SILENT 
Reapina. Tus Numsers Here REPRESENT HUNDREDTHS BUT ARE 
GIVEN AS WHOLE NuMBERS TO Avorn THE USE oF DECIMALS 








Day of drill | C. V. | Dayofdrill | C. ¥. 
1 26 15 19 
2 23 16 21 
3 23 17 18 
4 20 18 23 
5 12 19 14 
6 23 20 15 
7 22 21 20 
8 22 22 15 
9 26 23 14 

10 18 24 16 
11 19 25 16 
12 24 26 12 
13 26 27 14 
14 17 28 15 














TaBLE II.—SHOWING THE RESULTS OF THE SECOND EXPERIMENT. 


TuHE CogEFFI- 


CIENTS OF VARIATION ARE AGAIN GIVEN AS WHOLE NUMBERS TO AVOID 
THE UsE or DECIMALS 








Day of drill | C. V. | Dayofdrill | C.V. 
1 24 24 16 
2 23 25 16 
3 21 26 15 
4 17 27 14 
5 16 28 14 
6 22 29 14 
7 19 30 13 
8 18 31 15 
9 18 32 16 

10 18 33 15 
11 18 34 16 
12 17 35 16 
13 17 36 16 
14 17 37 13 
15 17 38 16 
16 18 39 16 
17 17 40 15 
18 17 41 5 
19 16 42 18 
20 17 43 15 
21 15 44 18 
22 15 45 16 
23 15 
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The purpose of this was to determine the changes in homogeneity as the 
group was practiced in a rather simple skill. The students were 
asked to make as many marks as possible in one minute. These were 
to be made in groups of five, with the left or wrong hand. The drill 
periods were forty-five in number and extended from October 14 to 
November 4 in the fall of 1929. 

The general trend of the C. V.’s is downward, the changes being 
very similar to those found in the first experiment. The first C. V. is 
twenty-four, the last is sixteen, which is only two-thirds as great. 
After the first twelve trials very few of the coefficients of variation are 
as much as three-fourths as great as the first two or three. The results 
of the two studies would seem to indicate that drill in either a broad or 
narrow mental function would cause the individuals in a comparatively 
homogeneous group to either remain so, or to become more and more 
alike in their achievement. 
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DISCUSSIONS 


STATISTICS NEVER EQUIVOCATE 


Conrad, in a recent article,! attempts to show that the distrust of 
intelligence ratings arising out of the subjective character of these 
ratings is unwarranted. He thinks he has done so by showing that 
the correlation of test scores with ratings is not affected materially by 
the “correction” of these ratings made by the army psychologists. 
Obviously Conrad has been misled by the equivocal language used by 
these psychologist. He quotes as follows: 


. . « If we place all individuals rated ‘‘6”’ for example, in the same class in our 
contingency table, whether they are all rated by the same person or not, the 
computed measure of relationship (between ratings and test scores) will be affected 
by factors other than those relevant to the problem (viz., by the “‘ personal equa- 
tion’) . . . It is necessary, therefore, to equalize the mean ratings of all groups 
before they are combined. This may be done by treating the class intervals on 


the rating scale (of each judge) as intervals on the axis of abscissae of a normal 
probability curve... 


Now it should be obvious that this so called equalization equalizes 
the judgment of different judges in range, and in nothing else. Thus if 
one judge tends to go to extremes and rate people either as very stupid 
or very brilliant whereas another is conservative and tends to place 
everyone at the golden mean, the army method would “equalize” 
such differences in temperament. If however one judge considers a 
certain characteristic, e.g., amenability to discipline, a sign of intelli- 
gence whereas another judge can discern intelligence, if present, in 
minor infractions of regulations, their judgments would not be equal- 
ized by the army method. There is in fact no possible method of 
“equalizing” such differences of opinion. They can only be compro- 
mised or averaged. In the special instance, where ratings of the same 
individuals by different judges are not available, even such averaging is 
impossible. 

With a large number of judges and ratings there is no apriori reason 
to expect an important change in correlation to arise out of the equal- 
izing adjustment of the army. Nothing whatever follows from the 
fact that such a change does not occur. The fact that the army 
psychologists have created the impression that they have corrected for 





1The Bogey of the ‘‘ Personal Equation” in Ratings of Intelligence. Journal 
Educational Psychology, February, 1932. 
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the unreliability of personal opinion is merely an illustration of the 

equivocation so common in the literature. The unreliability of 

personal opinion is not a bogey, however inconvenient that fact may be 

to those who try to achieve scientific progress by a determined pull at 

their boot straps. Curt ROsENow. 
Institute of Child Guidance, New York City. 





“THE EFFECT OF THE 6-22-44-22-6 NORMAL CURVE SYSTEM 
ON FAILURES AND GRADE VALUES’”—A COMMENT 


In a recent issue! J. DeWitt Davis uses the improved basis for a 
five-point grading system which I proposed in an earlier issue? to show 
that ‘‘a strict application” of it ‘‘in a twelve term course would elimi- 
nate practically fifty-one per cent of the matriculated group,” and that 
“‘the values of grades from term to term” would have “‘little directly 
comparable meaning.’’ His mathematical and statistical procedure 
seem to be entirely sound and his conclusions inevitable, once his 
fundamental assumptions are granted. That these assumptions are 
valid, however, may properly be questioned. 

1. Any division of the area under the normal probability curve may 
be made, and none necessarily implies any per cent of “failures.” 
The lower six per cent in the system which I suggested simple means 
‘distinctly inferior’ in comparison with the remainder of the group. 
An institution or an individual instructor or department may or may 
not decide that part or all of this inferior group should be ‘‘failed.”’ 
In many institutions both conditions and failures are included in this 
group. Yet the conditions and sometimes the failures may be made 
up. They do not necessarily imply permanent failure and elimination 
from school. 

2. Even more fundamental, however, is Mr. Davis’s assumption 
that failure in a single course automatically results in elimination from 
school at the end of the term in which it occurs. The average student 
takes from three to five courses; regulations in institutions vary widely 





1 Davis, J. DeWitt: The Effect of the 6-22-44-22-6 Normal Curve System on 
Failures and Grade Values. Journal of Educational Psychology, Vol. XXII, No. 8, 
November, 1931, pp. 636-640. 

2 Kells, Walter Crosby: An Improvement in the Theoretical Basis of the Five 
Point Grading System Based on the Normal Probability Curve. Journal of 
Educational Psychology, Vol. X XI, No. 2, February, 1930, pp. 128-136. 
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as to the number of these which must be failed to result in probation or 
complete disqualification. At Stanford University, for example, 
according to the latest published report of the Scholarship Committee, 
the disqualifications for the autumn quarter, 1929-1930, were fifty-nine 
in an undergraduate student body of two thousand six hundred ninety- 
six, or approximately two per cent, although the number of conditions 
and failures in separate courses was much higher. This assumption 
that a condition or failure in one subject any quarter automatically 
disqualifies a student from school or college is the fundamental fallacy 
that invalidates Mr. Davis’s major conclusions. 

3. On the assumption that elimination would take place on the 
foregoing basis, Mr. Davis constructs a nomograph and a table to 
show that sophomore grades would not be at all equivalent to freshman 
ones, and junior and senior grades decreasingly so. Grades received 
at these different levels, however, are not measures of a constant initial 
ability, but are measures of achievement at the different levels. 
Assume a normal distribution of grades in a freshman class in trigo- 
nometry. Probably a sophomore class in calculus would all be above 
the median of the trigonometry class, if marked on the basis of a 
comparable examination in trigonometry; but the sophomore grading 
should be adjusted to the level of sophomore achievement in the 
particular subject. An ‘‘A” grade in calculus would mean that a 
student was among the upper six per cent of the average students 
taking calculus. He should not be judged by a freshman trigonometry 
standard. The senior should be able to secure superior grades in 
most if not all freshman subjects. He is not therefore entitled to 
“A” grades in every subject. Inferiority or superiority in college 
grading can hardly mean with reference to a fixed standard; it must be 
relative. One is superior or inferior with reference to his peers in the 
same field. 

4. Incidentally Mr. Davis is in error when he says with reference to 
the suggested 6-22-44-22-6 system that practically one hundred per 
cent is included within +3 sigma range. The absolute error is not 
serious, but the principle is important. One of the distinctive theoret- 
ical as well as practical advantages of the system which I proposed was 
that it makes no such arbitrary assumption of the length of the base 
line of the normal probability curve. An assumption of a range of 
+3.0 sigma leads to a standard of 3.45 per cent of “A” grades, while 
a slight change in this assumption to +2.5 sigma leads to a standard of 
6.06 per cent instead—a striking difference for such a slightly differ- 
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entassumption.! Oneofthe great virtues of the suggested 6-22-44-22-6 
system is that it makes no such arbitrary assumption but includes the 
entire range of the normal probability curve as well as assuring the 
equality of the distances between the means of the successive grade 
groups. WALTER CrosBy EELLS. 


School of Education, Stanford University, California. 


1 Rugg, Harold O.: ‘‘Statistical Methods Applied to Education.”” Houghton 
Mifflin Co., Boston, 1917, pp. 217-218. 
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BOOK REVIEWS 


Norma V. ScHEIDERMANN. Psychology of Exceptional Children. 
Pp. XVII + 520. New York: Houghton Mifflin Co., 1931. 


As Cubberly says in his introduction to Psychology of Exceptional 
Children, ‘‘there are thousands of children in our schools today who are 
realizing but a small percentage of their educational possibilities.’ 
Miss Scheidermann’s book is an effort to present in a single volume 
pertinent available material concerning the atypical child, whether 
he be afflicted with deafness, blindness, feeble-mindedness, or what-not. 
It is a valiant attempt, but predestined to superficiality by the very 
great scope of her subject. 

Presumably the author did not have space for a careful critical 
evaluation of many of the investigations to which she refers; otherwise, 
it is difficult to understand how she could offer, without adverse com- 
ment, such generalizations as those of Murchison. (See quotation 
on page 458.) In discussing the relationship between church member- 
ship and criminality, Murchison has made such dogmatic statements as 
‘Methodists and Hebrews are superior as plain thieves,”’ offering as 
proof a series of unreliable percentages based, in some instances, upon 
fewer thana dozen cases. Data and quotations from such questionable 
sources appear altogether too often in Miss Scheidermann’s book, 
while there is a frequent failure to present the conclusions arrived at by 
men who are outstanding in their field. For example, Garth is not 
even referred to in the section on intelligence and race . . . And this 
reviewer is still puzzling over the possible justification for giving more 
space to a discussion of the left-handed child than to either the mentally 
subnormal or the mentally gifted. 

In spite of its many weaknesses, it is felt that Psychology of Excep- 
tional Children should be given a place as a reference book, since Miss 
Scheidermann has here made easily accessible a number of facts con- 
cerning the child who deviates in any particular from the normal. 

HERBERT A. CARROLL. 
University of Minnesota. 


JAMES Dunuap Licxitey. The Nervous System. Pp. XII + 144. 
New York: Longmans, Green and Company, 1931. 


In the new edition of The Nervous System Dr. Lickley has incorpo- 
rated some new material but in scope, style and organization there is no 
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change from the previous edition. It is a clear and concise introduc- 
tory handbook of the anatomy and physiology of the nervous system 
intended for non-medical students. The dominant level of descrip- 
tion is morphological; physiological facts are included merely to 
clarify the details of structure. There are one hundred seventeen well 
selected and well presented illustrations. Most of these are from 
17th and the 24th editions of Gray’s Anatomy. There are ten chapters 
in the book. In the first two the nature of nerve cells and nerve fibres 
are considered. In the last two localization areas and the sense organs, 
respectively, are treated. The remaining chapters are devoted to a 
description of the structure and function of the various parts of the 
nervous system. The work of Lashley, Child and Herrick are not 
considered. Here and there the author adds a sentence or two suggest- 
ing newer findings. The very simplicity and conciseness make exten- 
sive approaches impossible. Hence in his discussions the author 
wisely adopts the method indicated in his treatment of theories of 
color vision. Without mentioning other theories he prefaces a brief 
discussion of Hering’s theory, with a statement which says in effect that 
the one he selected seems to cover many of the known facts. The book 
contains an index. H. MELTZER. 
Psychiatric Child Guidance Clinic, St. Louis. 


Dovetas A. THom. Normal Youth and Its Everyday Problems. 
Pp. 367. New York: D. Appleton & Co., 1932. 


In Everyday Problems of the Everyday Child Dr. Thom achieved 
recognition for his excellent treatment of the problem of normal 
children in the first decade of life. His second book considers the 
behavior problems of normal adolescents and keeps up the excellent 
standard which he set in the first. He combines with his broad experi- 
ence a gift for clear and interesting exposition. 

For parents, teachers, and others interested in youth, Dr. Thom 
says he has attempted ‘“‘to discuss some of the pitfalls and hazards 
that surround the adolescent and in so far as possible, to suggest how 
they may be avoided; or, in those cases where difficulties already exist, 
to indicate in a general way the approach most likely to be helpful in 
the solution of the problem.” And the everydayness of the problems 
should be stressed here; the normal, not abnormal, youth is the center 
of the picture. There are those who will find in this book the first 
explanations of some of their own adolescent difficulties, but whether 
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that is true or not, it is my feeling that the book cannot be read without 
increased tolerance and concern for youths of high school and college 
age. 

For the beginner, abstract discussions of behavior problems are 
difficult to apply to concrete cases. Dr. Thom has avoided this 
objection through the inclusion of numerous definite illustrations of 
the type of problem discussed. They are succinct, omitting extraneous 
details, and lend a reality to the discussion which most readers will 
appreciate. 

If a reviewer may be permitted a bias, it leans to the chapters on 
Adjustment to Maturing Sex Drives, and Education. Physically the 
book is attractive, well-bound and contains an index. 


J. H. CoLeman. 
Huntington, N. Y. 


L. T. Trotanp. The Principles of Psychophysiology, Vol. III. Pp. 
XVII + 446. New York: D. Nostrand and Co., 1932. 


Cart Murcuison, Editor. A History of Psychology in Autobiography, 
Vol. II. Pp. 407. Worcester: Clark University Press, 1932. 


D. H. Kuup, II. Educational Sociology. Pp. XII + 591. New 
York: Longmans, Green and Co., 1932. 


A.J. Nocx. The Theory of Education in the United States. Pp. 160. 
New York: Harcourt, Brace, 1932. 


Here we have four most significant and interesting books on psy- 
chology and education. The third volume of Dr. Troland’s monu- 
mental work on psychophysiology is bound to be welcomed by all 
psychologists with gratitude, as have the two previous volumes. 
All one can do is express gratitude to the author for a difficult job 
magnificiently done. 

The second volume of the Clark University Press series of profes- 
sional autobiographies of continental and American psychologists of 
note makes as fascinating reading as did its predecessor. Students 
of the sciences need just this personal touch, this acquaintance 
with the human beings behind science, their struggles, failures and 
triumphs in the disinterested search for truth, to make them realize the 
human significance of science and to whet their appetite for making 
themselves familiar with its accomplishments. The Clark University 
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Press is doing psychology, and science in general, a great service in 
sponsoring such an enterprise. 

Dr. Kulp overworks the term scientific and makes the reader weary 
with ninety-nine pages of repetitions that education has not been, 
should be, and could be scientific. A brief introductory chapter would 
have been quite sufficient to make his point. There is hardly any need 
or justification for belaboring those educators of the past who theorized 
on the nature and function of education when they had no scientific 
data on which to base a science of education. And from the way Dr. 
Kulp uses the term philosophy to embrace everything towards which he 
is prejudiced, one might suspect that his conception of philosophy is 
hardly scientific. But once Dr. Kulp has worked his spite out of his 
system he makes a most significant contribution to the sociological 
aspect of education. His book should attract the closest attention 
of students of both education and sociology. 

To the reviewer, A. J. Nock’s book of one hundred sixty pages gives 
the most suggestive, succinct and illuminating survey and analysis of 
the theoretical basis of education in the United States now in print. 
Mr. Nock offers no panaceas for our educational ills, for which one is 
grateful to him. But he does point out that the three fundamental 
ideas in our educational theory, namely, equality, democracy, and the 
bearing of literacy upon public order and honest government, stand in 
need of careful clarification and modification. No one interested in 
education can afford to miss this most readable and honest book. 

Max ScHOEN. 
Carnegie Institute of Technology. 


Marion Monroe. Children Who Cannot Read. Pp. XVI + 205. 
Chicago: The University of Chicago Press, 1932. 


Evidence accumulated during the past ten years in the field of read- 
ing disabilities indicates that schools can no longer neglect their 
responsibility to the non-reader. With techniques now available all 
cases of reading deficiency may be partially or fully remedied providing 
that they are not entirely due either to lack of mental ability or to 
constitutional emotional deviations. 

The present monograph, an outgrowth of the previous interests of 
the author, is an important addition to the literature on diagnostic 
and remedial reading. The report consists essentially of methods of 
quantitative and qualitative measurement of reading ability, a discus- 
sion of causative factors, and results of remedial treatment. Four 
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hundred fifteen children with special reading defects were compared 
with a control group of one hundred one. It is unfortunate that the 
control group used was of superior intelligence in comparison to the 
normal or sub-normal experimental group since, in general, reading 
ability is markedly influenced by mental ability. Monroe’s earlier 
method of diagnosis has been amplified somewhat in the present study. 
A reading index, derived by dividing the reading grade (achievemert) 
by an average grade placement involving (1) chronological age, (2) 
mental age, and (3) arithmetical ability. It is difficult to understand 
why arithmetical accomplishment is included in this reading index. 
In fact, as shown by the two illustrations cited on p. 14, an average of 
grade placements derived from mental and chronological ages is almost 
identical with the average when arithmetic is included. The unneces- 
sary use of the arithmetical score is born out by the author’s own con- 
clusions in which no mention is made of arithmetic in listing functions 
upon which progress under remedial instruction depends. 

The chapter on “causative factors in reading defects” is the least 
adequate part of a book which is highly commendable on the whole. 
In this section the interpretations of the author have been influenced 
perhaps by certain points of view which need to be modified in the 
light of inferences which may be derived from a comprehensive analysis 
of contributions to the psychology of reading. An example is the 
assumption that left-eyedness fosters (1) excessive regressions in read- 
ing and (2) reversals in perception of letters and single words. One is 
more justified in inferring that both left-eyedness and tendencies to 
sinistrad sequencies in word and letter perception are due to cerebral 
dominance. Eye movements tend to be guided by perceptual tend- 
encies rather than vice versa. 

“The rate of progress in reading under remedial instruction was 
found to be a function of the child’s intelligence, his age, the number of 
hours spent in training, the number of months during which treatment 
was continued, the severity of the disability, the personality and 
behavior difficulties encountered in applying the remedial training, and 
the closeness of supervision of the remedial techniques. ”’ 

This book will be welcomed by educators and clinical psychologists. 
Diagnosis and remedial treatment are clearly stated and have proved 
practical and effective for ninety-eight per cent of all cases tutored 
under close supervision made normal or accelerated progress in reading. 

Mies A. TINKER. 


University of Minnesota. 
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Der La Fontaineriz. French Liberalism and Education in the Eight- 


eenth Century. Pp. VII + 385. New York: McGraw-Hill Book 
Company, 1932. 


This book contains the translations of the following essays on 
education in France in the eighteenth century: La Chalotais’ Essay on 
National Education, Turgot’s Memorial on Education, Diderot’s Plan 
of a University for the Russian Government, and Condorcet’s Report 
on the General Organization of Public Instruction. The authors of 
these essays register a complaint against the deplorable condition of 
education in France of that period. Chalotais centers his attack on the 
clergy. He wants education to be the function of the State. Turgot 
pleads for equality of educational opportunity. Diderot’s polemic is 
almost a satire on educational conditions in France, although it was 
written as a plan of a university for Russia. He decries the undue 
emphasis on the study of the Latin language and classical literature at 
the expense of the practical sciences. Condorcet, like other liberal 
reformers of his day, does not conceal his animosity toward the church 
and the classics, although his essay indicates that he proposed a milder 
means of reforming education than his contemporaries. He urges the 
State to assume the responsibility of training its citizens in civic duties 
and in moral obligations to society. 

If one were not familiar with the circumstances which so profoundly 
influenced Rousseau’s writing of Emile, he might have occasion to 
believe that the writers of the essays on education compiled by Mr. De 
La Fontainerie were inspired through some personal or selfish motive. 
But such was not the case. They had witnessed and deplored the 
abuses and misuses of educational practice by the churchmen of their 
time, and were thus inspired to suggest a remedy for the educational 
ills of the period. Their sole aim was to free the masses of people 
from the bondage to eccliasticism and to offer possible measures for a 
State-controlled system of education. Aside from the prejudices and 
oddities with which the essays are, in part, characterized, the discourse 
contains some very constructive criticism of educational practice, even 
in modern times. 

The reviewer believes that the usefulness of the book might have 
been.improved by introducing, at certain points in the translations, a 
few contrasts and comparisons with corresponding educational prac- 
tices inour own schools. A few parallelisms might have been indicated 
for the sake of students in courses of history of education without 
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detracting greatly from the organization of the book, and possibly 
without disrupting the discourse excessively. 

The translator has made a valuable contribution to the entire field 
of education, for there is much in the modern theory of education 
anticipated in the work. Students of education, teachers, supervisors, 
and administrators will find this compilation of essays very inspira- 
tional and helpful in their study and service. The book is translated in 
simple and effective English with an introduction extensive enough to 
furnish the ordinary reader with a good background of the historical 
facts incident to the study and understanding of the essays. 


RoBerT G. Smpson. 
Carnegie Institute of Technology. 


G. A. Yoakam. The Improvement of the Assignment. Pp. VII + 398. 
New York: Macmillan Company, 1932. 


One is impressed at his first glance into this book by the amount of 
useful information which the author has supplied on an apparently 
limited subject. Yet it would not be an exaggeration to say that it 
would have been possible to have included considerably more material 
of a very important character in the book. The subject is a worthy 
one for such a thorough treatment as the author has given it. 

The Improvement of the Assignment covers an extensive range of 
activities involved in the teaching and learning processes. Some of the 
more important ones which are considered in the book are: the impor- 
tance of the assignment, the need of improved assignments, the char- 
acteristics of a good assignment, the conditions essential for making 
successful assignments, the motivation of the assignment, the prepara- 
tion of new-type assignment materials, and the evaluation of the 
assignment. These important phases of the assignment are discussed 
in connection with the experimental evidence which the author has 
been able to obtain from time to time in his classes and seminars, and on 
the basis of evidence collected from studies and investigations which 
have been reported in research bulletins, educational periodicals, and in 
other sources. 

The author furnishes much information which clears up many of the 
controversal points of making assignments. He is careful in drawing 
conclusions from the research data which he has occasionally used and 
in suggesting reforms which cannot be, in part, supported by good 
practice. His discussions are not merely based on personal opinion. 
They are usually substantiated by such objective data as are available 
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in educational literature. He is conservative in expressing his opinion 
on matters which are, as yet, controversal. One of the most important 
contributions which Dr. Yoakam makes is this objective point of view 
in his book. 

A special feature of the book is the Chapter dealing with practical 
suggestions on the preparation of new-type assignment materials. A 
technique, together with illustrations, models, and useful suggestions 
for the guidance of the beginner in the writing of assignments, is 
indicated. This is the first treatment of the preparation of new-type 
materials in any book with which the reviewer is familiar. 

The book is explicitly written in good English and is very easy to 
read. It should be found an invaluable source of information for those 
who are engaged in planning and organizing activities and in directing 
study. RoBERT G. SIMPSON. 

Carnegie Institute of Technology. 


SAMUEL W. HarTWELu. Fifty-five ‘‘Bad’’ Boys. Pp. XVIII + 359. 
New York: Alfred A. Knopf, 1931. 


In this book the present director of the Worcester Child Guidance 
Clinic describes and interprets his psychotherapeutic endeavors with 
fifty-five difficult behavior problem boys. The cases reported repre- 
sent a chronological series studied and treated by the author at the 
Judge Baker Foundation during ‘‘a certain period of five months.’’ 
The introduction, by Dr. William Healy, is, therefore, in order. The 
content is presented under three divisions. Part One consists of 
three short chapters describing the problem, the plan, and technique 
respectively. Almost three-fourths of the pages are devoted to a 
description of the boys. This is done in Part Two. The case studies 
are frequently presented under such appealing captions as ‘‘Pleasures 
from Delinquency,’’ ‘“‘Distorted Lives,’’ ‘‘Misunderstanding.’”’ The 
three very short chapters in Part Three include a rough statistical 
summary and evaluation of the cases and outcomes. 

Individuals in search of profound analysis of deeper meanings 
underlying behavior, as orthodox psychoanalysts talk of such things, 
will find practically none of it in this book. Neither will they find 
obscurantism demonstrated. The case studies are not purported to be 
models for social workers to follow. Characteristically the boys are 
described in a manner which beginning students in a course in mental 
hygiene have described as concise, clear and intelligible. Some well 
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trained psychiatrists who expect more from published case studies than 
stereotyped presentations which follow a pre-determined pattern of 
thinking about, thinking with and thinking for every case find the 
descriptions, tiresome and interpretations, illogical. In reading the 
book one does, at times, hit spots which seem to justify Dr. Healy’s 
characterization of the author as possessing a “rugged honesty” as 
well as the qualities necessary for a sympathetic understanding of 
human beings. But, more often than not, it is the kind of sympathetic 
understanding and ruggedness which the reviewer associates with the 
experiential background of “‘educational” camp directors rather than 
with psychiatric or psychological learning and training. That is, 
with advisors who talk of the need for giving oneself rather than under- 
standing oneself and whose concern is with “‘making”’ or “breaking” a 
boy rather than understanding him and maintaining a learning attitude 
which will make further understanding possible. 

A feature which will interest interviewers is the discussion of degrees 
of rapport to adopt in working with different boys. The range 
described is from “friendly belief” to ‘‘dependent attachment.’’ The 
two intermediate degrees are labelled personal trust and “personality 
contact.”” In his treatment of this topic the author fails adequately to 
consider mood or temperamental differences in the same individual in 
different sessions and sometimes in the same session. 

H. MELTZER. 
Psychiatric Child Guidance Clinic, St. Louis. 
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